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Abstract 


In  May  2001,  the  Accelerating  Software  Technology  Adoption  (ASTA)  Initiative  of  the  Soft¬ 
ware  Engineering  Institute  (SEI)  piloted  a  workshop  to  support  the  management  of  technol¬ 
ogy  transition  and  to  capture  feedback  from  adopters  of  the  Capability  Maturity  Model®  Inte¬ 
gration  (CMMI^'^)  Product  Suite.  The  workshop,  titled  “The  Road  to  CMMI:  What  Works, 
What’s  Needed?,”  served  as  both  a  tool  for  improving  the  transition  of  the  CMMI  Product 
Suite  and  as  the  first  pilot  of  the  Technology  Transition  Workshop  (TTW)  series. 

The  workshop  was  a  success,  generating  75  prioritized  recommendations  of  what  works, 
what  doesn’t,  and  what’s  needed  for  an  organization  to  successfully  transition  to  the  CMMI 
Product  Suite.  Feedback  was  very  positive,  with  participants  indicating  that  the  workshop 
provided  beneficial  information  they  could  put  into  practice  at  their  home  organizations. 
Since  the  workshop,  ASTA’s  findings  have  been  disseminated  to  the  broader  CMMI  adopter 
population  through  the  CMMI  Technology  Conference  and  User  Group  held  in  November 
2001,  and  the  on-going  series  of  CMMI  Transition  Workshops  held  in  conjunction  with  the 
National  Defense  Industrial  Association  (NDIA).  The  findings  are  also  available  on  the  TTW 
Web  site:  http://www.sei.cmu.edu/products/events/ttw/.  Organizations  contemplating  the  im¬ 
plementation  of  the  CMMI  Product  Suite,  change  agents  for  those  organizations,  and  re¬ 
searchers  in  technology  transition  and  adoption  will  be  interested  in  this  report. 


®  Capability  Maturity  Model  and  CMM  are  registered  in  the  U.S.  Patent  and  Trademark  Office. 
CMM  Integration  and  CMMI  are  service  marks  of  Carnegie  Mellon  University. 


CMU/SEI-2002-TR-007 


IX 


CMU/SEI-2002-TR-007 


1  Workshop  Purpose 


1 .1  A  New  Technique  for  Managing  Transition 

In  January  2001,  the  Capability  Maturity  Model®  Integration  (CMMI^*^)  Product  Team  and 
Steering  Group,  who  are  responsible  for  developing,  supporting,  and  guiding  the  use  of  the 
CMMI  Product  Suite,  were  looking  for  ways  to  increase  interaction  with  the  emerging  CMMI 
user  community  to  learn  what  they  were  using  to  transition  to  the  early  versions  of  the  CMMI 
models.  They  also  wanted  to  know  what  mechanisms  users  felt  were  missing  that  would  be 
helpful  to  support  the  transition.  At  the  same  time,  the  Accelerating  Software  Technology 
Adoption  (ASTA)  Initiative  at  the  Software  Engineering  Institute  (SEI)  had  identified  a  need 
by  new-technology  developers  to  monitor  transition  progress  for  the  purpose  of  adjusting  and 
replanning  their  transition  strategies  and  methods.  Conversations  about  these  opportunities 
led  to  the  proposition  of  a  workshop,  which  eventually  became  the  Technology  Transition 
Workshop  (TTW),  that  would  bring  users  of  the  new  technology,  (i.e^,  the  CMMI  Product 
Suite)  together  to  discuss  their  experiences,  methods  being  used  in  their  implementations,  and 
gaps  in  their  approaches.  This  information  could  then  be  used  to 

•  provide  feedback  to  the  technology  developers  (in  this  case,  the  CMMI  Product  Team)  on 
the  state  of  their  technology’s  transition  into  the  community  so  that  they  could  leverage 
successes  and  plan  for  corrective  action  in  their  transition  efforts 

•  provide  an  environment  for  adopters  of  a  new  technology  (in  this  case,  the  CMMI  Prod¬ 
uct  Suite)  to  share  lessons  learned  from  their  adoption  efforts 

•  disseminate  knowledge  to  the  next  wave  of  technology  adopters  on  proven  enablers,  bar¬ 
riers,  and  gaps 

•  provide  information  to  potential  SEI  Transition  Partners  on  the  needs  of  the  CMMI 
community  that  the  partners  may  be  able  to  address 

These  are  important  for  accelerating  the  transition  of  an  emerging  technology  because  they 
enable  the  exchange  of  information  between  the  stakeholders  in  a  technology’s  value  net¬ 
work.  (See  Appendix  C  for  a  description  of  value  networks  and  their  importance  in  technol¬ 
ogy  transition.)  The  TTW  was  designed  to  bring  stakeholders  in  the  network  together  into  a 
workshop  environment  to  gather  information  that  is  relevant  to  the  other  stakeholders  in  the 
value  network,  including  the  CMMI  Product  Team.  This  critical  information  allows  develop- 


®  Capability  Maturity  Model  and  CMM  are  registered  in  the  U.S.  Patent  and  Trademark  Office. 
CMM  Integration  and  CMMI  are  service  marks  of  Carnegie  Mellon  University, 
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ers  of  new  technologies  to  apply  disciplined,  manageable  practices  when  transitioning  tech¬ 
nologies  into  widespread  and  routine  use. 

The  initial  workshop,  “The  Road  to  CMMI:  What  Works.  What’s  Needed?,”  was  proposed  to 
provide  the  CMMI  Product  Team  and  Steering  Group  with  information  about  the  user  com¬ 
munity  transitioning  to  the  CMMI  Product  Suite,  and  to  give  the  SEI  the  opportunity  to  de¬ 
velop  and  test  the  materials  for  an  ongoing  TTW  series.  The  ASTA  team  is  paying  close  at¬ 
tention  to  the  format,  instructions,  and  materials  for  future  workshops  so  that  the  capability  to 
plan  and  conduct  them  can  be  packaged  and  transferred  to  other  organizations  to  organize 
independently. 


1 .2  Statement  of  Purpose 

The  statement  of  purpose  for  the  initial  workshop  was: 

“The  pilot  workshop  will  equip  the  CMMI  Product  Team  and  Transition  Team 
with  community  experience,  captured  from  the  workshop,  that  contributes  to  the 
transition  success  of  the  CMMI  Product  Suite.  Attendees  will  share  and  learn 
from  each  other  and  receive  recognition  for  success.  ” 

ASTA  team  members  worked  closely  with  CMMI  Product  Team  members  throughout  the 
planning  and  organizing  stages  of  the  workshop  to  ensure  that  the  appropriate  activities  were 
planned,  the  appropriate  people  were  involved,  and  the  arrangements  would  achieve  the 
workshop’s  goals.  Information  about  the  workshop  can  also  be  viewed  on  the  TTW  Web  site: 
http://www.sei.cmu.edu/products/events/ttw/. 


1 .3  A  Survey  to  Jump-Start  the  Workshop 

Early  on,  we  decided  that  it  would  be  helpful  to  show  the  workshop  attendees  what  others 
were  doing  about  their  CMMI  transitions  by  collecting  survey  results  ahead  of  time  and  shar¬ 
ing  them  in  the  workshop.  We  designed  a  Web-based  survey  (available  on  the  TTW  Web  site 
at  http://www.sei.cmu/products/events/ttw/)  that  asked  CMMI  adopters  (including  prospec¬ 
tive  workshop  attendees)  about  their  CMMI  transition  efforts  and  experience.  The  survey  was 
accessible  to  anyone  visiting  the  workshop  Web  site. 


There  were  161  responses  to  the  survey.  Some  of  the  more  interesting  findings  include  the 
following: 

•  Organization  size  of  those  who  were  transitioning  to  the  CMMI  Product  Suite  ranged 
from  2  to  72,000  employees,  with  a  median  of  350  and  an  average  of  3,369. 
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•  Of  those  that  responded,  43%  stated  their  organization  had  already  made  the  decision  to 
adopt  the  CMMI  Product  Suite. 

•  About  half  of  those  who  were  committed  to  adopting  it  were  from  organizations  that  do 
both  systems  engineering  and  software  engineering.  The  other  responses  included  sys- 
tems-engineering  only  (3%),  software-engineering  only  (13%),  and  other  organizations. 

•  When  asked  about  the  motivation  for  transitioning  to  the  model,  a  common  theme  among 
those  who  answered  this  question  was  that  it  addressed  multiple  models  used  within  an 
organization  (“multi-discipline”)  and/or  was  required  by  customers  or  management  of  the 
organization  (“required.”)  Others  stated  that  it  provided  a  competitive  advantage  in  bid¬ 
ding  for  projects  against  other  organizations  (“competitive  edge”)  or  matched  a  corporate 
emphasis  on  Total  Quality  Management  (TQM)  or  other  quality  programs  (“quality.”) 

•  From  the  question  that  asks  about  “expected  benefits,”  “multi-discipline”  and  “quality” 
were  the  significant  keyword  leaders  and  seem  to  indicate  that  the  designed  multi¬ 
discipline  characteristics  of  CMMI  are  attractive  to  some  users,  and  that  quality  is  impor¬ 
tant  and  is  expected  as  a  benefit  from  using  CMMI. 

•  The  models  the  organizations  are  migrating  from  include  SW-CMM,  IS09000,  “other,” 
ISO/IEC  12207,  and  the  Systems  Engineering  CMM  (SE-CMM).  Six  companies  listed 
early  versions  of  the  CMMI  Product  Suite  under  the  “other”  category.  Most  of  the  re¬ 
sponding  organizations  are  currently  using  two  or  more  models. 

•  When  asked  what  strategy  they  are  using  for  implementing  the  product  suite,  nearly  half 
of  the  respondents  are  either  “not  sure”  of  their  plan  or  have  “no  plan  yet.”  Of  those  with 
a  plan,  “CMM  leverage,”  “study/leam,”  “assessments,”  and  “business-based  prioritiza¬ 
tion”  were  the  most  frequently  cited  strategies.  Although  they  did  not  explain  these 
strategies  in  detail,  about  half  of  those  who  will  leverage  other  CMM  models  said  that 
they  plan  to  finish  a  specific  CMM  maturity  level  before  transitioning  to  the  CMMI 
Product  Suite.  “Study/leam,”  “assessments,”  and  “business-based  prioritization”  may  in¬ 
dicate  that  those  who  responded  will  integrate  their  transition  efforts  into  existing  process 
improvement  projects,  depending  on  their  business  needs.  Only  7  of  the  161  respondents 
stated  they  had  a  detailed  transition  plan. 

•  One-half  of  the  people  who  responded  to  the  survey  provided  duration/effort  estimates 
for  their  transition  efforts.  Compared  to  the  source  CMM  models,  where  experience  has 
shown  that  it  takes  an  average  of  18  to  30  months  to  move  from  one  maturity  level  to  the 
next,  those  that  responded  expected  to  take  an  average  of  18  months  and  88  person- 
months  of  effort  for  the  transition. 

•  There  were  only  six  responses  to  a  question  about  which  CMMI  representation  they 
would  use,  staged  or  continuous.  It  appears  from  the  few  responses  here,  and  from  nu¬ 
merous  text  answers  in  the  “other”  response  for  several  questions,  that  organizations  have 
not  yet  decided  between  the  two  representations. 

•  Reasons  given  for  using  the  staged  representation  include 

-  “Senior  managers  and  customers  understand  this  easier.” 

-  “We  have  conducted  our  own  informal  survey  among  our  current  and  potential  cus¬ 
tomers,  as  well  as  among  government  contracting  organizations.” 

-  “Based  on  what  we  are  hearing,  we  currently  do  not  see  a  market  for  the  continuous 
representation.” 

-  “At  first  view,  staged  seems  to  best  meet  our  needs.” 
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•  Reasons  given  for  the  continuous  representation  include 

-  “The  ability  to  choose  a  process  area  (PA)  based  on  business  objectives  in  the  con¬ 
tinuous  view.” 

-  “Our  approach  is  to  improve  capability  continuously  rather  than  reach  a  maturity 
level.” 

“  “Some  process  areas  are  important  for  us  and  some  are  not.” 

-  “May  include  both  representations  when  we  consider  the  ‘software  factories’  at  our 
geographically  separated  units.” 

•  Only  a  few  people  responded  to  a  question  about  barriers  they  have  already  encountered 
during  their  transition  to  the  models,  citing  the  learning  curve  for  fully  understanding  the 
CMMI  Product  Suite  and  dealing  with  the  transition  in  the  midst  of  other,  continuing  pro¬ 
jects, 

•  Anticipated  problems  included  organizational  change  management  issues  and  managing 
a  different  scope  of  improvement  for  the  new  models.  The  cost  of  investing  in  the  transi¬ 
tion  and  the  lack  of  information  about  the  value  of  the  new  models  were  described  as 
risks  to  sustaining  the  improvement  effort. 

The  picture  the  survey  paints  is  of  a  community  of  organizations  that  are  undertaking  the 
transition  from  one  of  several  source  models  to  the  CMMI  Product  Suite.  Some  have  detailed 
transition  plans  while  others  are  in  the  very  early  stages  of  testing  for  sponsorship  and  owner¬ 
ship  for  the  transition,  as  well  as  gathering  information  to  determine  the  best  first  steps. 


The  above  survey  information  was  described  to  the  workshop  participants  so  that  they  could 
assess  how  the  larger  community  felt  about  these  issues  before  they  began  to  work  on  their 
tasks  of  identifying  “What  works?”  and  “What’s  needed?”  for  CMMI  transition  in  their  par¬ 
ticular  cases. 
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2  Participants  and  Agenda 


2.1  Participants 

In  planning  the  workshop,  the  workshop  committee  needed  to  ensure  that  the  individuals  who 
attended  were  representative  of  the  stakeholders  in  the  CMMI  Product  Suite’s  value  network. 
Since  the  CMMI  Product  Suite  had  not  yet  developed  a  map  of  their  value  network,  we  dis¬ 
cussed  and  came  up  with  an  initial  set  of  stakeholders  in  the  network.  While  this  was  not 
complete  or  final,  it  was  deemed  sufficient  for  this  first  workshop.  The  stakeholders  included 

•  organizations  that  have  started  to  adopt/transition  to  the  CMMI  Product  Suite 

-  organizations  that  piloted  the  CMMI  Product  Suite 

-  Level  5  organizations  that  are  moving  from  a  CMM  model  (Software,  Systems,  Inte¬ 
grated  Product  Development)  to  the  CMMI  Product  Suite,  based  on  their  technology 
change  management  capability 

-  other  organizations  that  are  adopting/transitioning  to  the  CMMI  Product  Suite  who 
have  worked  through  one  or  more  stages  of  transition  (e.g.,  as  described  in  the  Initiat¬ 
ing,  Diagnosing,  Evaluating,  Acting,  Learning  [IDEAL*'^]  model) 

•  transition  partners  who  have  worked  with  adopting/transitioning  organizations 

-  first-hand  experience  with  implementing  the  CMMI  models  in  organizations 

-  personal  experience  with  supporting  more  than  one  stage  of  adoption/implementation 
(e.g.,  as  described  in  the  IDEAL  model) 

•  CMMI  developers,  both  SEI  and  non-SEI 

•  ASTA  and  other  SEI 

-  workshop  planners  and  project  team 

-  workshop  work  group  facilitators 


An  important  criterion  was  that  participation  was  limited  to  those  organizations  that  are  al¬ 
ready  implementing  a  transition  process.  It  was  a  requirement  that  any  of  the  organizations 
that  were  invited  were  already  transitioning  to  one  of  the  models  from  a  CMM  background  or 
adopting  the  CMMI  Product  Suite  with  no  CMM  history. 


In  addition,  participants  were  asked  to  submit  a  short  white  paper  about  their  organization’s 
decision  to  adopt  the  CMMI  Product  Suite.  These  papers  are  presented  in  Appendix  A. 


IDEAL  is  a  service  mark  of  Carnegie  Mellon  University. 
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The  workshop  had  a  diverse  group  of  participants  with  varied  backgrounds  who  came  from 
the  Department  of  Defense  (DoD),  commercial,  and  transition  partner  domains.  Attendees 
included 

•  people  who  had  been  working  with  the  CMMs  for  years;  others  who  were  coming  into 
this  domain  for  the  first  time 

•  members  of  the  CMMI  Product  Team 

•  consultants  supporting  organizations  that  use  the  CMMs  and  the  CMMI  Product  Suite 

•  organizational  Lead  Assessors,  responsible  for  transitioning  to  the  Standard  CMMI  As¬ 
sessment  Method  for  Process  Improvement  (SCAMPI)  from  the  CMM-Based  Appraisal 
for  Internal  Process  Improvement  (CBA-IPI) 

•  Software  Engineering  Process  Group  (SEPG)  leaders  who  were  guiding  the  use  of  the 
CMMI  Product  Suite 

•  senior  managers  active  in  sponsoring  and  planning  process  improvement 

One  of  the  participating  organizations  had  started  its  process  improvement  program  using  the 
CMMI  Product  Suite;  others  were  moving  or  had  moved  to  the  model  from  one  or  more  of 
the  legacy  CMMs.  At  least  one  of  the  organizations  was  implementing  the  model  to  coexist 
with  SW-CMM-based  improvement  programs. 

The  mix  of  participants  was  an  early  indicator  of  the  lively  discussions  that  emerged  during 
the  workshop.  Multiple  perspectives  were  offered  on  many  of  the  topics.  One  participant  later 
commented  that  the  range  of  experience  in  the  meeting  generated  many  new  and  interesting 
ideas. 


2.2  Agenda 

The  agenda  was  designed  to  solicit,  collect,  and  synthesize  the  experience  of  the  participants. 
Day  1  of  the  workshop  began  with  participants  introducing  themselves,  explaining  their 
CMMI  experience,  and  presenting  their  white  papers  to  the  group.  The  participants  were  then 
briefed  on  the  CMMI  transition  survey  findings  and  shown  an  overview  of  technology 
change  management  concepts.  The  latter  is  summarized  in  Section  3  and  the  complete  brief¬ 
ing  is  provided  in  Appendix  B.  Armed  with  this  information,  the  participants  then  approached 
the  first  working  group  task:  to  identify,  based  on  the  white  papers  presented  earlier  in  the 
morning,  transition  mechanisms  that  other  participants  were  using  to  move  to  the  new  mod¬ 
els.  Section  4  of  this  report,  “Workshop  Results,”  describes  these  mechanisms.  In  our  exer¬ 
cises,  all  mechanism  names  were  collected  whether  the  mechanisms  worked  or  not.  We  saved 
a  discussion  of  the  usefulness  of  the  mechanisms  for  the  next  day’s  workshop  sessions.  Each 
group  posted  the  names  of  their  mechanisms  on  a  wall  chart  according  to  the  stage  of  the 
adoption  curve  (please  see  Section  3  for  information  on  the  term  “adoption  curve”)  where  the 
mechanism  was  first  used.  Many  of  the  mechanisms  were  useful  for  several  of  the  adopter 


6 


CMU/SEI-2002-TR-007 


groups;  however,  a  rule  was  established  that  the  mechanisms  were  to  be  posted  in  the  adopter 
stage  where  they  were  used  first. 

This  first  day  of  work  was  followed  by  an  expedition  to  a  Pittsburgh  restaurant  for  dinner  and 
awards.  We  recognized  and  applauded  the  workshop  participants  for  their  contributions  and 
thus  for  helping  to  make  the  transition  to  the  CMMI  Product  Suite  easier  for  others. 


Table  1:  Workshop  Agenda,  Day  1 


Day  1 

Agenda  Topic 

Desired  Outcome 

Opening  Remarks 

Understanding  of  the  objectives  for  the  workshop 
and  participant  roles. 

White  Paper  Presentations 

Participants  are  familiar  with  the  main  points  in 
white  papers. 

White  Paper  Q&A 

Answer  any  questions  regarding  white  papers. 

CMMI  transition  survey  results 

Review  and  identify  CMMI  transition  mecha¬ 
nisms  from  the  CMMI  transition  survey. 

Technology  Transition  and  Change  Management  Briefing 

Provide  participants  with  a  definition  of  "Transi¬ 
tion  Mechanisms"  and  understanding  of  the 
transition  models  to  be  used  in  the  workshop. 

Session  1  Working  Group  -  Identify  Transition  Mecha^ 
nisms  for  CMMI:  What  Works 

Participants  identify,  provide  examples  and  as¬ 
sess  transition  mechanisms  found  in  white  pa¬ 
pers,  the  CMMI  transition  survey  and  the  CMMI 
Product  Suite. 

Session  1  Working  Group  -  Agreement  on  Transition 
Mechanisms  for  CMMI:  What  Works 

Discussion  of  each  transition  mechanism  identi¬ 
fied  and  placement  on  the  Transition  Mecha¬ 
nisms  Table  for  CMMI  by  participants. 

End  of  Workshop — Day  I 

Evening  Social  and  Dinner  at  LeMont 

Recognize  participant  contributions  to  the  suc¬ 
cessful  transition  of  CMMI. 

On  the  second  day  of  the  workshop,  the  participants  reviewed  the  lists  of  mechanisms  and 
identified  those  that  were  most  useful  (see  Section  4  of  this  report  for  details).  The  “What 
Works?”  list  of  mechanisms  was  prioritized  according  to  which  ones  were  perceived  to  be 
most  important  and  effective. 


Then  the  participants  broke  into  two  working  groups  and  brainstormed  mechanisms  they 
wished  they  had  but  that  were  not  mentioned  in  the  earlier  exercises.  The  “What’s  Needed?” 
mechanisms  were  prioritized  according  to  which  ones  address  the  most  important  gaps  and 
thus  are  the  most  important  to  develop. 

The  participants  then  compiled  a  list  of  approaches  to  CMMI  adoption  that  should  be 
avoided.  This  was  referred  to  as  “traps  and  timewasters,”  and  Section  4  of  this  report  de¬ 
scribes  this  list  in  detail. 
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Table  2:  Workshop  Agenda,  Day  2 


Day  2 

Agenda  Topic 

Desired  Outcome 

Session  1  Working  Groups — Transition  Mechanisms  for 
CMMI:  What  Works  &  What  Doesn't 

Participants  evaluate  the  Transition  Mechanisms 
Table  and  develop  a  list  of  recommended 
mechanisms  for  the  transition  of  CMMI 

Session  1  Working  Groups — Transition  Mechanisms  for 
CMMI:  What  Works  &  What  Doesn’t  Outbricfing 

Working  Groups  present  their  list  of  recom¬ 
mended  mechanisms  for  the  transition  of  CMMI 

Session  2  Working  Group — Identify  Transition  Mecha¬ 
nisms  for  CMMI:  What's  Needed 

Working  Groups  identify,  provide  examples,  and 
prioritize  transition  mechanisms  not  found  (miss¬ 
ing)  in  the  Transition  Mechanisms  Table.  Deter¬ 
mine  placement  of  missing  mechanisms  on  the 
Transition  Mechanisms  Table  for  CMMI.  Pre¬ 
pare  and  present  recommendations  briefing  of 
group  work. 

Session  3  Working  Group — Identify  Transition  Mecha¬ 
nisms  for  CMMI  that  don’t  work 

Define  and  prioritize  things  that  caused  or  nearly 
caused  transition  efforts  to  fail  or  to  be  delayed. 

Occasionally,  as  requested  during  the  workshop,  members  of  the  CMMI  Product  Team  or 
other  workshop  participants  gave  demonstrations  or  briefings  on  different  CMMI-related 
tools  and  topics.  During  the  final  afternoon  of  the  workshop,  members  of  the  ASTA  program 
described  their  technical  program  work  and  demonstrated  some  of  the  tools  under  develop¬ 
ment  to  enable,  support,  accelerate,  and  evaluate  progress  in  transitioning  new  technologies. 


Table  3:  Workshop  Agenda,  Day  3 


Day  3 

Agenda  Topic 

Desired  Outcome 

Review,  discuss,  and  expand  on  the  results  from  all  of  the 
sessions. 

Review  and  discuss  mechanisms  presented  in 
each  of  the  sessions. 

Closing  Remarks 

Review  of  workshop  activities  and  next  steps  for 
the  CMMI  community. 

End  of  Road  to  CMMI  Workshop 

ASTA  Demonstrations 

Demonstrations  of  SEL/ASTA  technologies  cre¬ 
ated  to  assist  with  the  diffusion,  transition,  and 
adoption  of  new  technologies. 

Overall,  the  planned  agenda  was  followed,  with  a  few  improvisations  along  the  way  based  on 
the  discussions  and  ideas  that  developed  during  the  working  sessions.  We  interpreted  this  ad¬ 
aptation  to  be  a  healthy  sign  that  the  workshop  participants  felt  empowered  to  participate  and 
to  take  up  new  ideas. 
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3  Technology  Transition  Concepts 


The  workshop’s  introductions  and  white  paper  presentations  set  the  stage  for  a  discussion  of 
“transition  mechanisms.”  However,  before  beginning  to  describe  those  mechanisms,  we  pre¬ 
sented  workshop  members  with  an  overview  of  technology  change  management.  This  presen¬ 
tation  established  a  common  vocabulary  about  the  management  of  technology  change  that 
would  be  used  throughout  the  workshop,  and  it  introduced  the  categories  we  would  use  later 
when  collecting  names  of  transition  mechanisms.  The  presentation  with  detailed  notes  is  in¬ 
cluded  as  Appendix  B. 

As  we  sought  a  solution  to  the  workshop’s  objective  of  identifying  mechanisms  that  early 
adopters  think  would  improve  and  facilitate  an  organization’s  transition  to  the  CMMI  Product 
Suite,  we  provided  participants  with  some  fundamentals  about  the  common  complexity  of 
technology  adoption.  For  example,  when  managers  in  an  organization  decide  to  change  an 
organization’s  technology  processes,  they  face  some  typical  problems.  As  organizational 
processes  tend  to  be  stable  and  are  rarely  radically  changed,  managers  may  not  have  the  skills 
to  organize  and  plan  for  technology  change.  Moreover,  every  organization  is  different  and  its 
unique  skill  sets,  maturity,  and  characteristics  must  be  taken  into  consideration.  For  example, 
introducing  a  new  calendaring  system  into  an  organization  may  impact  managers  and  others 
who  coordinate  events  in  their  daily  work;  introducing  a  new  Integrated  Development 
Environment  (IDE)  will  impact  the  developers  who  must  use  it,  but  not  their  managers  who 
don’t.  The  bottom  line  is  that  each  new  technology  imposes  a  different  footprint  on  the 
organization,  and  needs  to  be  treated  uniquely.  However,  there  are  common  principles  that 
apply  to  the  adoption  of  technology.  If  organizations  can  address  technology  change  system¬ 
atically,  they  can  learn  to  master  it. 

Workshops  such  as  this  one  are  a  tool  that  can  be  used  to  facilitate  the  deployment  of  a  new 
technology.  These  workshops  provide  a  means  to  communicate  with  early  adopters  of  a  tech¬ 
nology,  to  learn  which  parts  of  the  whole  product  that  the  technology  suppliers  provided 
along  with  the  technology  were  useful  and  which  are  clearly  missing. 

The  TTW  series  provides  an  opportunity  to  test  whether  early  technology  adoption  planning 
is  working.  It  can  be  provided  after  the  early  adopters  have  engaged  with  the  technology,  be¬ 
fore  the  early  majority  have  become  engaged,  and  can  help  the  technology  sponsors  under¬ 
stand  what  it  will  take  to  get  the  technology  across  the  “gap”  between  early  adopters  and 
early  majority  users. 
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Understanding  the  need  for.  use  of,  and  design  of  this  workshop  depends  on  certain  ideas 
about  how  to  accelerate  technology  transition.  In  the  following  sections  we’ll  discuss 

•  adopter  populations  and  how  and  why  we  characterize  technology  deployment  by  the 
needs  of  the  adopters 

•  what  a  “whole  product”  is  and  what  it  means  for  technology  transition 

•  what  technology  “transition  mechanisms”  are 

•  the  commitment  curve,  what  it  is,  and  how  it  is  used 

The  briefing  at  this  point  in  the  workshop  covered  these  ideas  to  prepare  for  the  subsequent 
exercises  in  the  workshop. 


3.1  Adopter  Populations 

Work  by  Rogers  [Rogers  95]',  expanded  upon  by  Moore  [Moore  91).  describes  typical  cate¬ 
gories  of  technology  adopters.  The  total  adoption  population  is  described  as  a  normal  distri¬ 
bution  with  a  vertical  axis  for  the  number  of  adopters  and  the  horizontal  axis  for  time  inter¬ 
vals. 


Figure  1:  Adopter  Populations 


Moore  describes  five  categories  of  adopters,  Innovators,  Early  Adopters,  Early  Majority,  Late 
Majority,  and  Laggards,  as  follows: 

•  Innovators:  technology  enthusiasts  who  like  being  the  first  to  make  something  new  work. 

•  Early  Adopters:  visionaries  who  have  the  insight  to  match  an  emerging  technology  to  a 
strategic  opportunity,  the  temperament  to  translate  that  insight  into  a  high-visibility,  high- 
risk  project,  and  the  charisma  to  get  the  rest  of  their  organization  to  buy  into  that  project. 


'  This  work  was  originally  published  in  1962,  and  was  recently  reprinted  in  1995. 
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•  Early  Majority;  these  pragmatists  represent  the  bulk  of  the  technology  market.  They  are 
not  risk  takers  and  only  accept  a  technology  after  it  has  proven  itself.  They  prefer  retum- 
on-investment  information  and  want  to  know  how  others  have  fared  with  the  technology. 

•  Late  Majority:  conservatives  who  may  seem  to  be,  in  essence,  against  discontinuous  in¬ 
novations.  They  buy  and  use  technology  not  because  of  any  real  belief  in  the  technology 
but  because  they  feel  they  must  just  to  stay  on  a  par  with  the  rest  of  the  world. 

•  Laggards:  skeptics  who  do  not  participate  in  the  high-tech  marketplace  and  will  often 
block  innovations  to  maintain  the  status  quo. 

A  number  of  books  and  reports  describe  the  characteristics  of  these  five  categories  of  adopt¬ 
ers.  From  the  point  of  view  of  people  who  want  to  encourage  the  adoption  of  a  technology, 
getting  the  technology  adopted  by  the  early  majority  is  what  can  make  a  technology  a  suc¬ 
cess.  Innovators  and  early  adopters  provide  feedback  to  shape  the  technology;  however, 
unless  the  technology  provider  anticipates  and  meets  the  needs  of  the  early  majority  the  tech¬ 
nology  is  unlikely  to  make  it  across  the  “gap”  into  majority  use. 


3.2  Whole  Products 


There  are  a  number  of  ways  to  anticipate  the  needs  of  the  early  majority  users  of  a  technol¬ 
ogy.  The  ASTA  Initiative  at  the  SEI  provides  workshops,  technology-planning  approaches 
such  as  Transplant,  and  Web-based  adoption  tools  such  as  the  IDEAL-Based  New  Technol¬ 
ogy  Roll-out  (INTRo)  to  help  individuals  and  teams  anticipate  the  needs  of  adopters  to  sup¬ 
port  rapid  adoption  of  a  new  technology.  Among  the  things  that  these  tools  and  workshops 
support  is  the  identification  of  whole  product  components — those  things  that  need  to  be  built 
to  accompany  a  new  technology  to  meet  the  needs  of  the  early  majority  [Moore  91].  Whole 
product  elements  include  training,  case  studies,  instructions,  process  guides,  or  tools.  Figure 
2  below  illustrates  these  components.  By  anticipating  and  providing  the  components  that  the 
early  majority  adopters  will  need  to  adopt  the  new  technology  easily  and  quickly,  the  tech¬ 
nology  owners  can  accelerate  the  adoption  of  their  technology. 


Figure  2:  A  Whole  Product  Example  ^ 


^  Based  on  Fagan  Software  Inspections  as  implemented  at  AT&T  Bell  Labs. 
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3.3  Transition  Mechanisms 

The  components  of  a  whole  product,  besides  the  technology  itself,  are  transition  mechanisms. 

“A  transition  mechanism  is  the  means  by  which  information,  procedures,  or  skills  are 
communicated.  The  first  category  is  information  dissemination.  Examples  range  from 
marketing  brochures  and  advertising  to  engineering  handbooks.  The  second  category 
is  technology  implementation,  where  the  objective  is  to  alter  attitudes  or  behavior, 
including  new  skill  sets.  Examples  here  include  training  courses,  revised  reward 
systems,  and  policy  change”  [Fowler  93]. 

A  technology  transition  workshop  is  most  useful  after  a  technology  has  been  introduced.  It 
identifies  transition  mechanisms  by  building  on  the  initial  set  provided  with  the  technology’s 
introduction.  The  innovators  and  early  adopters  will  use  the  mechanisms  initially  provided 
and  will  experiment  to  find  out  which  combinations  work  most  effectively.  The  workshop 
allows  the  technology  owners  to  meet  with  those  early  users  (innovators  and  early  adopters) 
to  learn 

•  which  of  the  initial  mechanisms  worked  well 

•  mechanisms  that  didn’t  work  or  weren’t  used 

•  other  things  that  are  needed  (with  the  caveat  that  things  that  innovators  and  early  adopters 
want  may  not  appeal  to  members  of  the  early  majority  users) 

•  how  effective  their  initial  mechanism  planning  was,  so  that  future  planning  will  be  more 
effective 

The  workshop  focuses  on  transition  mechanisms — successful,  unsuccessful,  and  missing. 
This  information  can  then  be  used  as  input  for  replanning  for  majority  adoption.  Some  of  the 
missing  mechanisms  that  are  identified  may  be  built;  or,  the  information  from  the  workshop 
may  indicate  areas  for  further  research. 


3.4  Commitment  Curve 

It  is  not  enough  to  only  discuss  mechanisms,  or  those  components  that  worked  to  help  people 
adopt  a  technology  such  as  the  CMMI  Product  Suite.  Planning  when,  how,  and  why  to  apply 
the  mechanisms  is  critical  to  successful  transition.  When  transitioning  technology  to  an 
adopter  population  of  innovators,  early  adopters,  early  majority,  late  majority,  and  laggards, 
individuals  gain  commitment  to  the  use  of  the  technology  in  stages  that  are  often  depicted  as 
lying  along  an  S  curve,  as  in  Figure  3  below  [adapted  from  Conner  82].  This  commitment 
curve  describes  an  individual’s  commitment  (the  vertical  axis)  to  a  technology  increasing 
over  time  (the  horizontal  axis).  Stages  in  that  curve  are  identified  as  contact,  awareness,  un¬ 
derstanding,  trial  use,  limited  adoption,  and  institutionalization. 
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Pattenson-Connor  Change  Adoption  Model 
and  the  enabling  mechanisms  needed  to  support  Transition 
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Figure  3:  The  Commitment  Curve 


Each  member  of  an  adopter  population  progresses  through  these  stages  unless  they  get  stuck 
or  decide  to  abandon  the  technology  along  the  way.  When  the  aggregate  of  adopters  in  an 
organization  have  gone  through  these  stages,  we  consider  the  new  technology  institutional¬ 
ized  and  used  by  the  majority  segments  of  the  organization. 


These  stages  of  commitment  are  useful  for  planning  the  organization’s  transition  to  the  tech¬ 
nology.  For  example,  the  mechanisms  needed  to  create  awareness  probably  create  excitement 
rather  than  explain  the  technology,  whereas,  the  mechanisms  created  to  support  trial  use  may 
describe  how  to  identify  a  pilot  user  and  how  to  evaluate  the  pilot  results.  The  “what  works?” 
and  “what’s  needed?”  mechanisms  identified  by  workshop  participants  were  categorized  into 
the  stages  of  the  commitment  curve.  This  enabled  the  users  of  the  workshop’s  results  to  more 
clearly  understand  the  use  (or  intended  use)  of  the  mechanisms  and  the  importance  of  having 
mechanisms  at  every  stage  of  the  commitment  curve.  Stages  without  mechanisms  are  barriers 
to  successful  technology  adoption. 
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4  Workshop  Results 


The  workshop  participants  generated  the  following  three  categories  of  recommendations  for 
those  attempting  to  implement  the  CMMI  Product  Suite  and  those  that  support  or  enable  the 
implementation  such  as  change  agents,  the  CMMI  Product  Team,  and  SEI  Transition  Part¬ 
ners: 

1 .  60  recommended  practices  for  adopting  the  CMMI  Product  Suite 

2.  40  mechanisms  still  needed  to  improve  adoption 

3.  30  “traps  and  timewasters”  to  avoid 

These  130  separate  findings  and  recommendations  were  prioritized  and  the  73  most  impor¬ 
tant,  as  voted  on  by  the  participants,  include 

•  22  recommended  practices 

•  31  needed  mechanisms 

•  20  traps  and  timewasters 

These  findings  and  recommendations  should  enable  future  adopters  to  plan  and  execute  more 
effective  technology  transitions  to  the  CMMI  Product  Suite.  They  also  provide  valuable  input 
to  the  rest  of  the  CMMI  community,  including  potential  SEI  Transition  Partners  who  may  use 
this  information  as  preliminary  research  on  which  to  base  their  development  of  CMMI  transi¬ 
tion  products  and  services. 


4.1  Recommended  Transition  Mechanisms 

Based  on  their  experience,  the  workshop  participants  recommended  the  mechanisms  for  fu¬ 
ture  adopters  that  are  listed  in  the  following  table.  The  mechanisms  were  categorized  into  one 
of  the  five  phases  of  adopter  commitment  discussed  in  Section  3.  Note  that  mechanisms  may 
be  used  in  several  of  the  phases,  but  were  categorized  in  this  workshop  according  to  the  phase 
of  their  first  use.  For  example,  in  the  following  table,  the  mechanism  “Multiple  communica¬ 
tion  channels  established”  is  helpful  in  several  of  the  stages  of  commitment,  but  its  first  use  is 
recommended  in  the  Contact  and  Awareness  stage. 
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Table  4:  "What  Works"  Mechanisms 


Stage  of  Commitment 

Mechanism 

Votes 

Contact  and  Awareness 

Think  CMMI:  reference  cards:  promotional  materials 

14 

SEI  material  translated  into  local  language 

8 

Multiple  communication  channels  established 

4 

CMMI  awareness  briefings/forums  held 

3 

Understanding 

Methods  for  self-assessment;  gap  analysis;  mini-assessments;  class 
B&C  assessments  that  relate  gaps  to  the  organization’s  processes 

20 

Chart  that  describes  process  responsibilities  of  different  roles/across 
organizational  boundaries 

11 

Poster  on  CMMI 

7 

Transition  roadmap  for  the  organization 

7 

CMMI  implementation  action  plans 

4 

Birds  of  a  Feather  sessions  on  focused  topics 

4 

Trial  Use 

Integrating  measurement  of  PI  progress  into  QA 

8 

Linking  of  the  QA  process  to  CMMI 

8 

Transition  strategy  SW-CMM  CMMI 

8 

Pilot/trials  in  non-SW-development  areas 

7 

Example  CMMI  process  improvement  budget 

5 

Limited  Adoption 

Role-based  training 

24 

23 

Transition  steering  group 

10 

ROI  trend  data 

9 

Integrating  all  disciplines  into  the  process  group 

8 

Institutionalization 

CMMI  best-practice  based  templates/checklists/assets 

22 

Integrating  process  review  into  project  management  reviews 

14 

4.2  A  Wish  List  of  Future  Mechanisms 

In  the  second  stage  of  the  workshop,  the  participants  compiled  a  wish  list  of  transition 
mechanisms  that  they  would  like  to  add  to  their  arsenal  of  mechanisms  outlined  in  the  previ¬ 
ous  section.  The  table  below  shows  the  results  of  the  prioritization  and  the  number  of  votes 
that  each  of  these  mechanisms  received. 
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Table  5:  "WhaVs  Needed"  Mechanisms 


Stage  of  Commitment 

Mechanism 

Votes 

Contact  and  Awareness 

Widely  published  list  of  organizations  who  have  decided  to  transi¬ 
tion,  to  use  for  source  selection 

21 

A  clear  vision  of  what  CMMl  is 

11 

Integrated  product  suite  across  the  adoption  spectrum,  a  “whole 
product” 

11 

Well-written  PR  materials  targeting  senior  managers,  project  manag¬ 
ers,  and  system  engineers 

10 

Web-based  mechanism  linker  (e.g.  Amazon.com  model  that  recom¬ 
mends  mechanisms  you  might  want  based  on  those  you’ve  used  or 
asked  about) 

8 

Clear  and  unambiguous  statement  from  DoD  on  what  their  intentions 
are  for  using  CMMI  for  both  government  and  contractor  organiza¬ 
tions 

8 

Statistically-based  information  to  demonstrate  that  benefit  is  being 
derived  from  using  CMMI 

7 

Understanding 

Technical  sales  pitch,  describing  the  promise  of  CMMI 

10 

Project-supported  class  C  assessment  method 

9 

Rewritten  CMM  TRs  for  CMMI  usage,  especially  reports  about 
measurements  and  metrics 

7 

Expert  evaluation  of  implementation  and  artifacts  for  CMMI 

5 

CMMI  version  of  Mastering  Process  Improvement  and  PCM  Method 

3 

Guidelines  for  using  core/common  PAs  in  areas  not  covered  by 

CMMI 

2 

Trial  Use 

Tailored  transition  guides  for  different  transition  paths 

8 

. 

Guidelines  for  establishing  and  mapping  the  organization’s  process 
architecture  that  are  consistent  with  organization’s  culture  and 

CMMI 

6 

Mapping  of  organizational  roles  to  CMMI  goals  and  practices 

6 

Project-supported  class  B  assessment  method 

7 

Guide  for  designing  pilots  to  maximize  adoption 

5 

Organizational  mentoring  program 

5 

ROI  calculator 

4 
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Stage  of  Commitment 

Mechanism 

Votes 

Limited  Adoption 

Adoption  guide  organized  by  organizational  characteristics  (domain, 
size,  market) 

8 

DoD  CMMI  assets  library'  (policies,  procedures,  sharing  mecha¬ 
nisms) 

7 

Materials  to  enable  people  to  become  CMMI  subject  matter  experts 
(e.g..  black  belt) 

6 

Training  “starter  kit”  with  training  design  content  and  role-specific 
training  recommendations 

6 

Modem/altemative  training  mechanisms  for  CMMI  (CBT,  VTC, 
Web-based) 

6 

Web  site  of  assets  needed  for  assessment-SCAMPI  materials 

4 

Method  and  technology  for  continuous  process  assessment/ 
evaluation 

4 

Institutionalization 

Strategic  plan  from  CMMI  Steering  Group  for  transition  and  product 
evaluation 

5 

Incorporation  of  CMMI  content  into  Defense  Acquisition  University 
curriculum 

5 

Guidelines  for  negotiating  interfaces  based  on  customer  and  supplier 
relative  process  maturities 

3 

Certification  process  for  CMMI  assessments 

3 

4.3  Traps  and  Timewasters 

This  section  of  the  workshop  prompted  energetic  responses  from  the  workshop  attendees. 

The  purpose  was  to  capture  lessons  learned  about  what  not  to  do  to  transition  an  organization 
to  the  CMMI  Product  Suite.  We  had  no  idea  when  we  planned  this  session  that  this  would 
turn  out  to  be  so  therapeutic  for  the  attendees.  This  list  was  not  categorized  into  the  five 
stages  of  commitment,  but  it  was  prioritized,  as  the  table  below  shows. 
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Table  6:  Traps  and  Timewasters 


Trap  or  Timewaster  V 

Have  Process  Engineering  Group  (PEG)/Software  Engineering  Process  Group  (SEPG)  meetings  I! 

with  no  project  representation 


Overdo  (e.g,,  write  100  page  procedure)  Risk,  M&A,  DAR  Process  Areas  when  going  from  SW- 
CMM  to  CMMl 


Don’t  link  process  to  product  quality,  cost,  schedule,  and  performance 


Rely  on  current  “Introduction  to  CMMI”  training  as  sufficient  for  assessment  team  training 
Let  experts/zealots  write  the  procedures 


Set  artificial  level  requirements,  and  put  the  people  with  the  lowest  estimate  in  charge 


Spend  most  of  your  time  on  the  open-ended  questions  during  a  SCAMPI  assessment 


Don’t  train-it  costs  too  much.  Just  do  it-follow  the  assessor 


Management  should  dictate  process  changes  without  any  coordination,  because  it  speeds  things  up 
Don’t  bother  to  capture  the  hearts  and  minds  of  middle  management 


Select  your  most  important  (i.e.,  crucial)  project  as  your  CMMI  pilot-get  biggest  bang  for  your  buck  8 
Change  the  organization  structure  6  months  before  the  assessment,  to  clarify  reporting  structures  8 


Include  zealots  in  specific  areas  (like  measurement,  international  standards)  in  your  assessment  team 


Tell  people  they  can  understand  the  model  just  by  reading  it 
Align  your  practices  exactly  to  the  CMMI,  instead  of  to  what  you  do 


Use  a  benchmark  method  (e.g.  Class  A  assessment)  for  first  contact 


Put  as  many  Lead  Assessors  on  your  assessment  team  as  possible.  Different  opinions  add  spice 


Forget  the  “I”  phase  in  the  IDEAL  model 


Use  the  Introduction  to  CMMI  course  as  first  contact  for  program  managers 


Rotate  your  SEPG  leader  every  three  months-use  someone  with  a  fresh  look  who  has  never  read  the 
policy! 
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5  Conclusions 


This  workshop  was  the  first,  pilot  offering  of  the  TTW  series  and  was  a  resounding  success. 
The  workshop  was  an  exploration  of  how  to  design  and  facilitate  future  workshops  to  achieve 
the  following  goals  in  the  support  of  transition  management: 

•  to  provide  feedback  to  the  technology  developers  (in  this  case,  the  CMMI  Product  Team) 
on  the  state  of  their  technology’s  transition  into  the  community  so  that  they  can  leverage 
successes  and  plan  for  corrective  action  in  their  transition  efforts 

•  to  provide  an  environment  for  adopters  of  a  new  technology  (in  this  case,  CMMI)  to 
share  lessons  learned  from  their  adoption  efforts 

•  to  provide  knowledge  to  the  next  wave  of  technology  adopters  on  proven  enablers,  barri¬ 
ers,  and  gaps 

•  to  provide  information  to  potential  CMMI  transition  partners  on  the  needs  of  the  CMMI 
community  that  the  partners  may  be  able  to  address 

For  the  first  goal,  the  feedback  from  the  CMMI  product  team  leader,  who  participated  in  the 
entire  workshop,  was  very  positive.  He  has  since  used  the  results  and  outputs  from  the  work¬ 
shop  in  multiple  venues  to  communicate  to  the  rest  of  the  CMMI  Product  Team,  the  Steering 
Group,  and  the  greater  CMMI  community  the  findings  of  this  early  CMMI  adoption  coiranu- 
nity.  CMMI  sponsors,  senior  managers,  and  key  stakeholders  have  had  several  opportunities 
to  receive  information  about  what  the  community  believes  it  needs  to  continue  and  accelerate 
CMMI  adoption.  These  results  have  been  inputs  for  reassessment  of  transition  plans,  whole 
product  development,  and  transition  support  for  the  CMMI  Product  Suite. 

For  the  second  goal,  the  feedback  from  the  workshop  participants  was  very  enthusiastic  and 
far  better  than  we  had  expected  for  a  pilot.  Participants  took  away  ideas  that  they  thought 
they  could  apply  in  their  organizations,  and  heard  about  barriers  and  time  wasters  that  they 
need  to  watch  out  for  as  they  continue  their  adoption  efforts. 

For  the  third  goal,  the  workshop  findings  have  been  disseminated  to  the  broader  CMMI 
adopter  population  through  the  CMMI  Technology  Conference  and  User  Group,  sponsored 
by  the  National  Defense  Industrial  Association  (NDIA),  in  Denver,  CO,  November  2001.  The 
first  CMMI  Transition  Workshop  was  held  in  Orlando,  FL,  January  17-18.  For  more  informa¬ 
tion  about  NDIA-sponsored  workshops,  please  see  http://www.sei.cmu.edu/cmmi/comm 
/ndia-workshop.html.  The  findings  are  also  available  on  the  TTW  Web  site: 
http://www.sei.cmu.edu/products/events/ttw/. 
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The  fourth  goal,  to  provide  information  to  existing  or  potential  Transition  Partners  about  op¬ 
portunities  for  providing  transition-enabling  mechanisms,  was  also  accomplished  at  the 
CMMI  Technology  Conference  and  User  Group  in  November  2001,  and  through  public  dis¬ 
tribution  of  this  report  on  the  TWW  Web  site. 


Overall,  the  workshop  design  used  in  this  pilot,  along  with  sufficient  flexibility  in  the  facilita¬ 
tion  process  to  allow  for  shifts  in  the  agenda  as  needed,  proved  to  be  a  successful  package.  A 
few  improvements,  solicited  from  the  participants,  will  be  integrated  into  the  next  TTW,  to  be 
conducted  sometime  in  2002.  These  improvements  include 

•  providing  (before  the  workshop  starts)  information  on  whole  product  mechanisms  identi¬ 
fied  and  released  by  the  technology  owners,  so  that  their  usefulness  can  be  explicitly 
commented  on  by  workshop  participants 

•  presenting  very  clear  criteria  for  the  participants  to  use  for  prioritizing  the  lists  that  they 
develop 

•  distributing  read-ahead  materials  on  technology  transition  concepts 

•  understanding  and  managing  perceptions  of  balanced  participation  of  the  various  stake¬ 
holder  communities 

During  2002,  the  lessons  learned  from  a  second  pilot  of  the  TTW  series  will  be  packaged  to¬ 
gether  with  guidance,  templates,  and  other  necessary  instructions  so  that  other  organizations 
and  transition  partners  can  implement  these  workshops  as  a  part  of  their  transition  manage¬ 
ment  processes. 
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Appendix  A  White  Papers 


This  appendix  presents  the  original  white  papers  submitted  by  TTW  participants.  The  follow¬ 
ing  table  summarizes  the  names  and  organizations  of  those  who  submitted  white  papers.  Not 
all  white  paper  authors  were  able  to  attend  the  workshop.  It  should  be  noted  that  these  papers 
have  not  been  edited  but  have  only  been  slightly  modified  for  presentation  within  this  report. 


White  Paper  Author(s) 

Home  Organization 

Jeffrey  Dutton  (attended) 

Sverdrup  Advanced  System  Group 

Geoffrey  Draper  (attended) 

Harris  Corporation 

W.  H.  Eyster  (attended) 

Craig  Hollenbach  (attended) 

Litton  PRC 

A1  Pflugrad  (attended) 

Kenneth  Funkhouser  (attended) 

Concurrent  Technologies  Corporation 

Mark  Campbell 

Ron  Ulrich  (attended) 

TRW,  Inc. 

Rick  Hefner 

SuZ  Garcia  (attended) 

aim  ware,  Inc. 

Wayne  Sherer  (attended) 

TACOM-ARDEC 

Mary  Gregg  (attended) 

Chuck  Gordon 

Alison  Ferraro  (attended) 

Bruce  Boyd  (attended) 

The  Boeing  Company 

Working  Group  1.5^ 

SEI  and  other  organizations 

Winifred  Menezes  (attended) 

Q-Labs,  Inc. 

^  This  white  paper  was  contributed  by  a  working  group  from  the  High  Maturity  Workshop  that  was 
held  at  the  SEI  in  March  2001.  Some  of  its  authors  participated  in  the  TTW. 
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Technology  Transition  Workshop 
The  Sverdrup  Experience 
WHITE  PAPER 

J.  Dutton 
May  21 , 2001 


INTRODUCTION 

The  400-person  Sverdrup  Advanced  Systems  Group,  consisting  of  a  Headquarters 
and  10  field  offices,  began  the  CMMI^'^path  with  the  CMMI-SW  V0.1  in  February  of 
1999.  We  are  scheduled  for  our  Level  3  assessment  against  the  Staged  version  of 
the  CMMI-SE/SW  on  August  20,  2001 . 

THE  SVERDRUP  LANDSCAPE 

Although  one  sister  organization-  the  Arnold  Engineering  Development  Center-  had 
made  significant  progress  toward  SW-CMM®  Level  2  compliance  when  ASG  started 
its  CMMI  effort,  and  although  our  Sverdrup  Technology  Headquarters  was  ISO  Reg¬ 
istered,  there  was  no  real  process  culture  evident  when  we  started.  Our  ASG  Field 
Offices  serve  all  four  military  Services  and  other  customers-  so  we  also  faced  signifi¬ 
cant  differences  in  technical  domain  and  customer  cultures.  The  “organization”  we 
are  taking  to  CMMI-SE/SW  Level  3  compliance  is  scattered  across  eight  states. 
What  this  means  is  that  we  had  to  find  a  way  to  ensure  we  were  fully  aware  of  and 
carefully  considered  each  office’s  requirements  for  standard  process  and  for  tailoring 
the  standard  process. 

We  brought  three  advantages  to  this  melee:  1)  Our  Vice  President  is  unwaveringly 
committed  to  the  CMMI  effort,  2)  our  corporate  parent,  Jacobs  Engineering,  has  de¬ 
veloped  mature  training  and  process  improvement  methods  that  we  are  using,  and 
3)  a  small  cadre  of  personal  who  have  in-depth  experience  with  the  SW-CMM. 

HISTORY  AND  ORGANIZATION  OF  THE  CMMI  EFFORT 

The  effort  to  adopt  the  CMMI-SW  V0.1  was  formally  kicked  off  in  February  of  1999 
with  the  official  formation  of  the  SEPG.  Our  initial  goal  was  to  reach  Level  3  compli¬ 
ance  by  April  of  2001 .  The  decision  to  migrate  to  the  CMMI-SE/SW  was  made  in  No¬ 
vember  of  1999.  The  Engineering  Performance  Improvement  Center  was  formed  the 
same  month  to  lead  the  effort  toward  systems  and  software  engineering  improve¬ 
ment.  We  had  an  initial  assessment  “profile”  in  April  of  2000,  and  have  had  two  more 
since. 

Our  assessment  date  was  recently  moved  out  three  months  to  August  20,  2001  by 
our  Engineering  Management  Council.  The  reasons  for  the  move  focused  on  improv¬ 
ing  the  quality  and  detail  of  our  processes  and  training  materials. 

The  Engineering  Management  Council  (EMC)  oversees  and  directs  the  CMMI  effort- 
including  acting  as  CCB  for  the  standard  process.  The  EMC  is  chaired  by  the  Vice 
President  and  General  Manager  of  Sverdrup  ASG,  and  its  members  are  the  Office 
Managers,  Directors,  and  others  representing  key  business  and  infrastructure  ele¬ 
ments. 

The  Engineering  Performance  Improvement  Center  (EPIC)  conducts  the  CMMI  im¬ 
provement  effort.  It  consists  of  a  Technical  Director,  an  Engineering  Process  Asset 
Library  (EPAL)  administrator,  and  seven  EPIC  Field  Office  Leads  chosen  and  funded 
in  a  part-time  capacity  by  the  larger  ASG  Field  Offices.  Creative  work  and  reviews 
are  accomplished  by  these  people  working  as  the  EPIC  Technical  Committee. 
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STRATEGIES 

In  retrospect,  we  have  put  in  place  five  separate  but  interdependent  strategies  for 
the  CMMI  effort: 

Standard  process  development  strategy 
Cultural  change  strategy 
Deployment  strategy 
Quality  Assurance  strategy 
Assessment  strategy 

The  most  significant  issues  to  address  in  the  standard  process  development  strategy 
were  1)  to  ensure  that  the  breadth  of  technical  domains  were  adequately  addressed, 
and  2)  to  ensure  that  our  tailoring  process  accounted  for  the  differences  in  technical 
domain  as  well  as  size  and  scope  of  projects.  Our  tailoring  approach  requires  the 
calculation  and  application  of  a  Tailoring  Guidance  Factor-  which  ensures  tailoring 
decisions  are  made  in  a  consistent  and  repeatable  manner. 

The  major  challenge  in  our  cultural  change  strategy  was  (and  is)  the  movement  of 
management  and  leadership  personnel  to  a  process-literate  point  of  view.  The  ar¬ 
ticulation  of  just  how  things  were  done  so  that  we  could  capture  them,  then  ensure 
they  were  CMMI-  compliant,  was  a  major  effort.  Getting  business  and  technical 
leaders  to  think  in  terms  of  doing  things  in  a  standard-  although  tailored-  manner  was 
and  is  a  tough  problem. 

Our  deployment  strategy  is  absolutely  key  to  our  success.  We  decided  to  deploy  the 
prototype  standard  process  to  four  Pilot  Projects  in  the  fall  of  2000,  and  have  used 
the  lessons  learned  to  complete  the  detail  and  revise  the  standard  process.  In  addi¬ 
tion,  we  have  elected  to  deploy  the  baseline  standard  process  to  a  controlled  num¬ 
ber  of  projects  across  ASG  to  ensure  continued  success  and  compliance.  At  the 
same  time,  all  ASG  engineering  and  engineering  management  personnel  receive  the 
entire  suite  of  training  courses-  so  that  they  are  prepared  to  implement  the  standard 
process  when  called  upon  to  do  so.  A  ‘lull  compliance”  date  is  being  set-  beyond 
which  all  applicable  projects  and  organizational  units  must  be  compliant  with  the 
CMMI  through  our  standard  process. 

The  quality  assurance  strategy  was  confronted  with  three  challenges:  1 )  ensure 
compliance  with  CMMI  and  standard  process  2)  encourage  and  support  improved 
processes  and  technologies  in  response  to  organizational  goals,  and  3)  find  a  way  to 
fund  both  of  these  in  a  funding-constrained  environment.  We  have  done  all  three. 

Our  assessment  strategy  was  put  in  place  to  conserve  assessment  funding  while 
reducing  the  risk  of  finding  unknown  problems  at  the  point  of  our  Level  3  assess¬ 
ment.  We  elected  to  forego  a  Level  2  assessment  in  favor  of  two  additional  profiles 
(ARC  Class  C  assessments). 

STANDARD  INTEGRATED  ENGINEERING  PROCESS  (SIEP) 

In  order  to  provide  a  sharper  focus  on  engineering  and  engineering  management 
processes  important  to  the  organization,  we  modeled  our  Integrated  Engineering 
Process  Architecture  on  the  ISO  12207  framework.  We  defined  12  core  processes, 
which  was  later  reduced  to  1 1 .  They  are  shown  below. 
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This  set  of  core  processes  exhibits  several  important  attributes.  It  helps  us  focus  on 
the  “mission”  processes  of  Project  Management  and  Product  Engineering.  It  sepa¬ 
rates  engineering  support  from  product  engineering-  and  it  provides  core  process 
“homes”  for  important  organizational  functions.  To  implement  these  core  processes, 
we  have  developed  six  elements  that  define  the  Standard  Integrated  Engineering 
Process.  These  are: 

•  Engineering  Performance  Improvement  Program  Plan  (EPIP  Plan)-  contains  the 
standard  process  for  process  development,  deployment,  and  management,  as 
well  as  the  current-year  plan  for  milestones,  schedule,  and  resources. 

•  Knowledge  Management  Plan-  contains  the  standard  process  for  knowledge 
management/  training  and  the  annual  plan  for  execution. 

•  Quality  Assurance  Plan-  contains  the  standard  process  for  quality  assurance  as 
well  as  the  annual  plan  for  the  QA  program. 

•  Measurement  and  Analysis  Plan-  contains  the  standard  process  for  metrics. 

•  Purchasing  Manual-  contains  the  standard  process  for  supplier  agreement  man¬ 
agement. 

•  Integrated  Engineering  Handbook-  contains  the  standard  process  for  Project 
Management,  Product  Engineering,  Engineering  Support,  and  most  engineering 
management  activities. 

IMPACT  ON  CULTURE  AND  INFRASTRUCTURE 

The  realization  that  the  CMMI  (any  version)  has  an  impact  on  EVERY  part  of  an  or¬ 
ganization  came  as  a  bit  of  a  shock.  Changes  to  our  training  organization,  to  our  IT 
group,  to  our  proposal  process-  and  a  technical  relationship  with  our  time-card  ac¬ 
counting  system,  are  examples.  We  are  evolving  from  an  organization  that  has 
grown  at  the  rate  of  30%  per  year  through  the  efforts  of  technical  and  business  lead¬ 
ership  to  one  that  will  now  follow  a  rigorous  engineering  and  management  model  to 
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do  the  same  thing  in  a  standard  and  repeatable  manner.  The  impact  has  been  and 
continues  to  be  significant. 

COMMUNICATIONS  CHALLENGES  AND  SOLUTIONS 

The  organizational  difficulties  associated  with  the  first-time  adoption  of  a  capability 
maturity  model  are  magnified  by  the  geographic  separation  of  our  field  offices.  Each 
major  office  has  a  fairly  unique  customer  base,  and  a  unique  set  of  engineering  and 
management  values  and  experiences. 

Communication  is  required  at  management  and  technical  leadership  levels  to  de¬ 
velop,  coordinate,  verify,  maintain,  implement,  institutionalize  and  improve  the  stan¬ 
dard  process  and  associated  technologies.  Our  technology  solution  is  our  Distributed 
Work  Environment,  discussed  below. 

The  organizational  communications  solution  is  based  on  the  EMC  and  EPIC,  and  the 
relationship  between  the  two.  In  order  for  our  CMMI  implementation  to  work,  each 
EPIC  Field  Office  Lead  must  lead  CMMI  adoption  in  their  location,  and  must  work 
closely  with  the  local  manager  to  ensure  all  local  concerns  are  satisfied. 

ENGINEERING  AND  MANAGEMENT  ARTIFACTS 

The  following  artifacts  are  used  by  the  indicated  parties: 

•  Action  Directive:  A  highly  structured  mechanism  for  assigning  actions,  resources, 
and  due  dates,  and  allowing  the  assignee  to  negotiate  and  commit  to  the  action 
as  planned  or  negotiated.  Follow-up  and  closure  is  required.  Used  by  Senior,  In¬ 
termediate,  and  Project  Managers. 

•  Defect  Removal  Form:  Used  to  document  and  track  to  closure  all  “in  phase” 
problems  in  documents  or  software.  Applied  during  Peer  Reviews,  Unit  Test,  and 
Integration  Tests  by  the  responsible  engineer. 

•  Problem  Report/Baseline  Change  Request:  Used  to  document  all  “out  of  phase” 
problems  with  baseline  products  or  documents. 

•  Standard  Operational  Project  Reviews:  Pre-formatted  monthly  reviews  by  each 
Project  Manager  to  the  Organizational  Unit  Manager.  Includes  applicable  project 
metrics. 

•  Process  Improvement/Feedback  Form:  Used  by  anyone  in  the  organization  at 
any  time  to  suggest  an  improvement  to  the  SIEP. 

•  Waiver  Request:  Used  by  Project,  Intermediate,  or  Senior  Managers  to  request  a 
waiver  to  training  requirements  or  to  the  implementation  of  the  SIEP. 

•  Knowledge  Base  Assignment  Form:  Used  as  agreement  between  each  engineer¬ 
ing  or  engineering  manager  and  his  or  her  supervisor  to  assign  duties  so  that 
training  courses  can  be  assigned  for  the  year. 

•  “Executable”  word  templates  for  all  engineering  plans  and  documents. 

•  Deployment  Plan  template:  Used  to  deploy  all  SIEP  documents  or  changes  to 
those  documents. 

•  Metrics  database  switchboard  and  forms 

TOOLS  AND  TECHNOLOGIES 

The  center-piece  of  the  Distributed  Work  Environment  (DWE)  is  a  data-video  confer¬ 
encing  capability  that  is  internet  accessible  from  all  of  our  locations  and  that  provides 
a  high  degree  of  information  protection.  The  DWE  also  includes  the  EPAL,  and  a 
configuration  management  system  for  process  engineering  documents.  The  DWE 
supports  the  following  functions: 
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•  Process  engineering  (EPIC):  process  development,  hosting,  and  configuration 
management.  Data-video  conferencing  and  advanced  CM  tools. 

•  EPAL:  Hosts  the  Engineering  Process  Asset  Library  for  organizational,  interme¬ 
diate  and  project  processes,  artifacts,  and  metrics. 

•  Training:  Supports  internet-based  distributed  training  and  testing. 

•  Reviews:  Supports  EPIC  and  Project  Management  reviews. 

TRAINING  APPROACH 

As  our  long  term  approach,  we  have  adopted  a  Knowledge  Management  approach. 
Year  2001  solutions  are  centered  on  a  more  traditional  training/testing  solution-  but 
we  plan  to  evolve  toward  a  more  active  KM  approach  during  the  latter  part  of  2001 . 

CMMI  CONCERNS 

Our  overall  experience  with  the  CMMI  has  thus  far  been  very  positive.  Our  concerns 
lie  in  an  area  already  well  known  to  the  community-  and  where  evolutionary  solutions 
are  being  trailed.  We  would  like  to  see  the  cost  of  assessments  and  evaluations  con¬ 
tinue  to  go  down  (both  to  the  organization  and  to  the  evaluating  organization  in  the 
case  of  evaluations).  We  also  believe  that,  as  both  assessment  and  evaluation 
methods  evolve,  strategies  and  methods  should  be  available  to  dramatically  reduce 
the  costs  associated  with  process  state  measurement  and  reporting. 

Required  information: 

Jeffrey  L.  Dutton 

Technical  Director,  Engineering  Performance  Improvement  Center 

Sverdrup  Advanced  Systems  Group 

4717  University  Drive,  Suite  101 

Huntsville,  AL  35816 

Phone  256-426-0324 

Email:  jeff.dutton@att.net  or  duttonjl@sverdrup.com 
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CMMI  Transition  at  Harris  Corporation 


Geoff  Draper 

Engineering  Process  Group 
Harris  Corporation 

Government  Communications  Systems  Divi¬ 
sion 

P.O.  Box  37,  M/S:  2/9702 
Melbourne  FL  32902 
gdraper@harris.com 
(321)727-5617 


W.  H.  Eyster 

Vice  President  -  Engineering 
Harris  Corporation 

Government  Communications  Systems  Divi¬ 
sion 

P.O.  Box  37,  M/S:  2/6500 
Melbourne  FL  32902 
heyster@harris.com 
(321)727-5500 


Process  Improvement  History 

CMMF"^  transition  at  Harris  Corporation  follows  a  lengthy  history  of  CMM-based 
process  improvement.  Harris  divisions  have  been  aligned  with  SEI  and  SW-CMM®- 
based  process  improvement  since  the  Harris  Information  Systems  Division  (HISD) 
first  became  a  SEI  software  process  affiliate  in  1989.  CMM-based  improvement  ini¬ 
tiatives  have  been  most  mature  in  Harris  government  contracting  divisions  (7  SPA  / 
CBA  IPI  SW-CMM  assessments  and  2  SE-CMM  assessments  across  4  divisions), 
but  are  also  used  as  an  improvement  framework  in  Harris  commercial  divisions  (1 
CBA  IPI). 


Following  a  Harris  restructuring  in  July  1999,  the  majority  of  the  company’s  U.S. 
government  contracting  business  has  been  consolidated  into  Harris  Government 
Communications  Systems  Division  (GCSD).  Most  recently,  GCSD  completed  a  CBA 
IPI  in  January  2000  revalidating  a  SW-CMM  Level  3  maturity  level  rating  achieved  by 
two  legacy  divisions  in  1994  and  1995.  Currently,  GCSD  is  actively  pursuing  a  SW- 
CMM  Level  4  rating  through  deployment  of  a  division-wide  measurement  initiative. 


Harris  has  no  direct  experience  with  EIA/iS-731 ,  but  has  been  using  its  predecessor 
SE-CMM  as  a  basis  for  systems  engineering  process  improvement  for  several  years. 
SE-CMM  assessments  were  performed  in  1 996  and  1 997  within  two  GCSD  legacy 
divisions. 

Similarly,  Harris  has  not  formally  used  the  IPD-CMM,  but  has  substantial  project  ex¬ 
perience  with  Integrated  Product  Team  (I PT)  environments,  and  intends  to  adopt  the 
IPPD  component  for  its  CMMI  transition  path. 

A  summary  of  the  GCSD  process  improvement  history  is  depicted  in  Figure  1 . 


CMMI  is  a  service  mark  of  Carnegie  Melion  University. 

©  Capability  Maturity  Model  and  CMM  are  registered  with  the  U.S.  Patent  and  Trademark  Office. 
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GCSD  Process  Improvement  Structure 

Specializing  in  the  areas  of  airborne,  spaceborne,  and  ground-based  communica¬ 
tions,  Harris  GCSD  develops  and  produces  information  processing  and  communica¬ 
tions  systems  to  collect,  store,  retrieve,  process,  analyze,  display,  and  distribute  in¬ 
formation  for  the  U.S.  Government,  its  agencies,  and  its  prime  contractors.  With  the 
consolidation  of  these  several  prior  divisions,  the  diverse  set  of  GCSD  application 
domains  includes  areas  such  as  space  and  ground  based  communications  systems, 
satellite  communications,  weather  systems,  image  processing  and  visualization,  sig¬ 
nal  processing,  optical  processing,  avionics,  phase  array  antennas,  and  precision 
structures.  Projects  include  a  variety  of  new  development,  R&D,  and  Operations  & 
Maintenance  (O&M)  programs,  which  range  from  COTS  integration  to  custom  hard¬ 
ware  and  software  development.  The  GCSD  engineering  community  consists  of  over 
2600  engineers,  including  systems  (500),  software  (500),  RF  /  analog  (250),  electri¬ 
cal  digital/network  (300),  signal  and  optical  processing  (150),  mechanical  (500),  lo¬ 
gistics  (200),  and  technicians  (200). 

This  diversity  in  engineering  disciplines  and  application  domains  has  been  a  primary 
consideration  in  the  GCSD  decision  to  adopt  the  CMMI-SE/SW/IPPD  model  as  a  ba¬ 
sis  for  engineering  process  improvement.  Cross-disciplinary  process  improvement 
has  been  a  Harris  practice  that  began  with  HISD  expanding  its  SEPG  into  an  Engi¬ 
neering  Process  Group  (EPG)  upon  achieving  SW-CMM  level  3  in  1994.  The  GCSD 
EPG  now  consists  of  8  engineering  discipline  teams:  Software,  Systems,  Electrical, 
Mechanical,  RF  /  Antennas  /  Optics,  Integration  &  Test,  Specialty  /  Support  Engi¬ 
neering,  and  Engineering  Information  Technology.  An  Engineering  Process  Council 
(EPC)  consisting  of  senior  engineering  management  and  staff  guides  GCSD 
engineering  process  improvement. 
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Short-Term  Tasks  Long-Term,  Persistent 


Engineering  Process  Council  (EPC) 

-  Executive  Steering  Group 

Engineering  Process  Group  (EPG) 

-  Coordinate  and  manage 
improvement  initiatives 

-  Impiement  cross-disciplinary  action 
teams 

Engineering  Discipline  Teams  (8) 

-  Systems  Engineering 

-  Software  Engineering 

-  Electrical  Engineering 

-  Mechanical  Engineering 

-  Antennas  /  RF  /  Optics 

-  Integration  &  Test 

-  Specialty  /  Support  Engineering 

-  Engineering  Information  Technology 


Figure  2.  GCSD  Process  Improvement  Structure 

Harris  CMMI  Transition  Strategy 

A  roadmap  describing  the  Harris  GCSD  transition  strategy  for  CMMI  adoption  is  de¬ 
picted  in  Figure  3.  A  brief  description  of  the  components  of  this  plan  is  contained  be¬ 
low. 


m^m 

Legacy  CMM-Based  Improvements 

_ ] 

n 

•  SW-CMM  /  SE-CMM  Action  Plans,  Command  Media  Integration,  Level  4  Initiative 

CMMI  Involvement  and  Preparation 

_ 1 

•  CMMI  Stakeholder,  PDT,  SCAMPI,  Training,  SEI  Transition  Partner,  Lead  Assessor 

Model  Selection  ]| 

Gap  Analysis  | 

Process  Updates  ^ 

Deployment  ] 

•  SE/SW/IPPD  (Staged) 

•  Process  Mapping 

•  Command  Media, 

•  Organization, 

•  Traceability 

Training,  etc. 

Projects 

[  CMMI  Action  Plans  | 

•  CMMI  Model  Components,  Issue  Closure,  Organization  Awareness,  Pilots 

f 

Mini-Assessments  | 

•  SW-CMM,  SE-CMM,  CMMI 

Appraisal 

I 

•  Summer/Fall  2001 

Figure  3.  Harris  GCSD  CMMI  Transition  Roadmap 
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Legacy  CMM-Based  Improvements 

CMMI  represents  a  natural  evolution  from  Harris  investment  in  legacy  systems  and 
software  process  improvement  models  (SW-CMM  v1.1,  SE-CMM  v1.0).  GCSD  has 
also  developed  an  internal  HW-CMM  for  structuring  hardware  process  improvement 
initiatives  in  selected  situations  with  strategic  business  value  or  customer  emphasis. 
As  part  of  its  ongoing  process  improvement  strategy,  GCSD  has  established  process 
improvement  action  plans  for  each  of  its  engineering  disciplines.  Based  on  years  of 
metrics  data  collection,  the  ROI  from  CMM-based  improvement  is  a  key  factor  attrib¬ 
uted  to  improvements  observed  in  engineering  productivity  and  quality. 

This  applies  not  only  within  the  government  market,  which  demands  a  demonstrated 
level  of  process  maturity  and  proven  performance  in  order  to  compete  for  new  busi¬ 
ness,  but  ongoing  CMM-based  improvement  and  transition  to  CMMI  is  also  planned 
within  certain  of  the  Harris  commercial  divisions  due  to  similar  business  needs  and 
market  demands. 

Even  though  CMMI  has  introduced  additional  process  areas  and  practices,  CMMI  is 
fundamentally  consistent  with  these  legacy  models.  GCSD  therefore  believes  that 
current  improvement  initiatives  based  on  these  still  make  strategic  business  sense, 
and  investments  will  migrate  naturally  to  CMMI.  Most  notable  in  this  regard  is  the 
ongoing  GCSD  SW-CMM  Level  4  initiative,  which  includes  deployment  of  a  division¬ 
wide  measurement  framework  and  integrated  metrics  database.  Recently  3  pilot  pro¬ 
jects  have  validated  via  mini-assessments  to  be  operating  in  accordance  with  Level 
4  practices,  and  the  pilot  program  is  being  expanded  to  4  additional  projects. 

CMMI  Involvement  and  Preparation 

Harris  has  long  been  involved  in  the  CMMI  project,  as  a  stakeholder  reviewer  since 
Augu.st  1998  and  Product  Development  Team  (PDT)  member  since  February  2000, 
and  currently  as  co-lead  of  the  CMMI  Assessment  Methodology  Integrated  Team 
(AMIT).  In  addition  to  helping  develop  a  process  improvement  framework  that  can  be 
viably  used  across  industry  for  internal  process  improvement  and  external  capability 
evaluations,  this  has  provided  GCSD  with  early  visibility  and  insight  into  the  details  of 
the  CMMI  models  and  how  they  can  be  used  effectively  to  strengthen  and  guide  im¬ 
provement  of  GCSD  engineering  processes. 

Harris  has  been  accepted  as  a  CMMI  transition  partner  for  internal  delivery  of 
SCAMPI  assessment  services  and  Introduction  to  CMMI  training  instruction.  Harris 
currently  has  one  SEI-authorized  CBA  I  PI  lead  assessor,  and  is  planning  to  seek  au¬ 
thorization  for  two  SCAMPI  lead  assessors. 

Model  Selection 

Harris  GCSD  intends  to  migrate  to  a  staged  representation  of  the  CMMI- 
SE/SW/IPPD  model,  due  primarily  to  market  preference,  legacy  investment,  and  fa¬ 
miliarity  with  the  SW-CMM  within  the  GCSD  engineering  and  management  commu¬ 
nity. 
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Harris  commercial  divisions  adopting  CMMI  intend  to  migrate  to  the  continuous  rep¬ 
resentation  of  the  CMMI-SE/SW  model,  due  to  its  flexibility  in  selecting  process  ar¬ 
eas  and  capability  levels  for  improvement  based  on  business  value.  However,  since 
several  of  these  organizations  are  early  in  their  process  improvement  lifecycle,  a  hy¬ 
brid  approach  is  also  recommended  using  the  equivalent  staging  of  the  continuous 
representation  for  initial  pursuit  of  staged  level  2  process  areas  before  migrating  to  a 
fully  continuous  representation  approach  for  focusing  ongoing  improvements. 

Gap  Analysis  and  Mini-Assessments 

The  GCSD  EPG  is  currently  utilizing  an  internal  mini-assessment  technique  to  de¬ 
termine  the  organizational  mapping  and  gap  analysis  as  part  of  its  transition  effort 
based  on  the  CMMI-SE/SW/IPPD  model.  The  Harris  mini-assessment  process  is  a 
consensus-based  approach  that  has  been  used  for  several  years  within  GCSD  and 
Harris  commercial  divisions  as  an  efficient,  effective,  and  low-cost  mechanism  to 
identify  strengths,  weaknesses,  best  practices,  and  improvement  opportunities  rela¬ 
tive  to  a  CMM-based  model.  A  description  of  this  mini-assessment  process  for  the 
SW-CMM  can  be  found  in  the  October  1 999  issue  of  Crosstalk 
(http://www.stsc.hill.af.mil/CrossTalk/1999/oct/natwick.asp),  but  essentially  provides 
traceability  from  CMM  practices  to  organizational/project  command  media,  and  as¬ 
signs  a  rating  (0-10)  for  each  practice  in  terms  of: 

Approach  -  the  extent  to  which  the  practice  is  described  in  organizational/project 
process  descriptions 

Deployment-  how  consistently  the  practice  is  implemented  and  institutionalized 
Results  -  the  effectiveness  and  positive  results  realized  from  deployment  of  the 
practice 

The  mini-assessment  process  has  been  adapted  for  CMMI  by  incorporating  each 
CMMI  specific  practice  and  generic  practice,  establishing  traceability  to  where  the 
practice  is  implemented  in  the  organizational  command  media,  identifying  sources  of 
objective  evidence,  and  assigning  ratings  in  each  of  these  dimensions.  This  effort  is 
being  implemented  initially  by  the  GCSD  Systems  Engineering  and  Software  Engi¬ 
neering  discipline  teams  (reference  Figure  2),  with  facilitation  by  the  EPG.  Results  of 
these  mini-assessments  are  being  used  to  establish  CMMI  action  plans  to  address 
the  assessment  findings. 

Initially  focused  only  on  SE  and  SW  disciplines  at  the  organizational  level,  the  CMMI 
transition  strategy  intends  to  extend  the  CMMI  mini-assessment  process  to  addi¬ 
tional  disciplines,  and  to  division  projects  upon  deployment  of  the  revised  CMMI- 
compliant  organizational  processes. 

The  Harris  CMMI  mini-assessment  process  can  be  characterized  as  an  ARC  Class 
C  method.  Based  on  several  years  of  use,  this  mini-assessment  has  proven  effective 
in  identifying  the  key  process  strengths,  weaknesses,  and  improvement  needs  for 
both  organizational  and  project  processes.  It  also  provides  an  excellent  indicator  of 
the  projected  results  for  future  internal  or  external  appraisals,  with  considerably  less 
effort  and  organizational  impact  than  formal  appraisal  methods. 

Based  on  this  experience,  Harris  recommends  that  other  organizations  consider  a 
variety  of  assessment  approaches  in  determining  their  CMMI  process  improvement 
path,  rather  than  relying  solely  on  SCAMPI  Class  A  assessments.  Class  B  and  Class 
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C  methods  can  be  a  much  more  cost-effective  approach,  with  relatively  little  com¬ 
promise  in  assessment  accuracy  or  basis  for  improvement  initiatives. 


CMMI  Action  Plans 

Findings  from  the  CMMI  gap  analysis  and  mini-assessments  are  used  to  identify  and 
prioritize  GCSD  improvement  initiatives.  In  accordance  with  its  defined  processes  for 
SW-CMM  Level  3,  GCSD  follows  its  Engineering  Process  Improvement  Handbook 
(EPIH)  to  plan,  execute,  and  manage  engineering  process  improvement  activities.  A 
transition  plan  is  used  to  establish  long-range  strategic  improvement  initiatives 
based  on  business  objectives,  internal  /  external  appraisal  findings,  and  internal  im¬ 
provement  requests.  These  goals  are  used  to  determine  prioritized  tactical  initiatives, 
developed  annually  with  the  EPG  and  EPC.  Individual  improvement  activities  are 
developed  by  the  respective  engineering  discipline  teams  or  EPG,  depending  on 
breadth  of  coverage,  with  overall  cost/schedule  management  provided  by  the  EPG 
leader. 

Several  mechanisms  are  being  used  to  increase  organizational  awareness  of  CMMI 
within  the  GCSD  engineering  and  management  community.  This  includes  delivering 
briefings  and  lunchtime  forums  on  the  CMMI  model  and  components,  intranet  web 
pages,  and  status  briefings  on  GCSD  CMMI  transition  and  improvement  initiatives. 

GCSD  has  also  printed  and  distributed  copies  of  a  pocket-sized  CMMI  Reference 
Card  to  its  engineering  and  management  population,  as  part  of  encouraging  the  or¬ 
ganization  to  ‘Think  CMMI”.  Softcopy  source  for  the  reference  card,  in  PageMaker 
format,  was  obtained  from  SEI. 

Although  Harris  is  a  transition  partner  for  delivery  of  Introduction  to  CMMI  training, 
the  primary  delivery  of  the  full  training  course  is  anticipated  to  be  as  part  of  SCAMPI 
assessment  team  training.  Typically,  Harris  does  not  offer  the  full  CMM-based  model 
training  to  its  general  engineering  community;  rather,  overviews  are  incorporated  as 
applicable  into  standard  engineering  process  training  courses  and  briefings.  How¬ 
ever,  additional  full  training  courses  may  also  be  offered  to  other  Harris  divisions. 

Process  Updates  and  Deployment 

Updates  to  the  organizational  command  media  are  planned  to  incorporate  the  im¬ 
provements  developed  by  the  GCSD  CMMI  action  plan.  This  includes  such  assets 
as  process  descriptions,  training,  checklists,  and  templates.  A  Configuration  Control 
Board  (CCB)  is  established  to  manage  improvement  requests  to  GCSD  process  me¬ 
dia  and  assets,  using  change  requests  documented  in  a  Problem  /  Improvement 
Tracking  Report  (PITR). 

Based  on  the  scope  of  the  improvement  activity,  piloting  is  typically  used  to  validate 
improvements  prior  to  organization-wide  deployment. 

Upon  incorporation  of  CMMI-related  improvements  into  the  organizational  process 
and  project  defined  processes,  the  GCSD  mini-assessment  process  will  again  be 
used  to  assess  the  implementation,  deployment,  and  results  of  CMMI  transition  ef¬ 
forts  at  the  organizational  and  project  level.  A  custom  database  is  being  developed 
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to  document  and  manage  findings  from  mini-assessments,  internal  SCAMPI  process 
improvement  assessments,  and  external  capability  evaluations. 

Appraisal 

Following  initial  implementation  of  the  GCSD  CMMI  transition  plan,  a  Class  A 
SCAMPI  assessment  is  planned  for  late  in  the  2001  calendar  year. 

Summary  Recommendations 

Based  on  experience  with  CMMI  transition  at  Harris  Corporation  to  date,  the  follow¬ 
ing  lessons  learned  and  summary  recommendations  are  offered  as  guidance  to 
other  organizations  pursuing  similar  initiatives: 

Start  early! 

•  Understand  the  differences  from  legacy  models  and  methods. 

•  Identify  gaps  and  establish  action  plans. 

•  Do  not  underestimate  the  organization  learning  curve. 

Continue  current  improvements  based  on  legacy  models 

•  Do  what  makes  sense  for  your  organization. 

•  Investments  will  migrate  naturally  to  CMMI. 

Consider  opportunities  to  implement  Integrated  engineering  assets 

•  Policies,  processes,  training,  metrics. 

•  Reinforce  with  templates,  checklists,  etc. 

Use  a  variety  of  assessment  methods  (Class  A,  B,  C  methods) 

•  Techniques  such  as  mini-assessments  can  be  effective  and  cost  efficient. 

•  Class  A  (SCAMPI)  assessments  may  not  always  be  the  most  appropriate  choice. 
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Litton/PRC,  McLean,  Virginia 


Maturity  Levei 
Date  of  Assessment 
Lead  Assessor(s) 

Point  of  Contact 
Web  Page 

Size  of  the  Organization 
Typicai  Program  Size 


Primary  Application  Do- 
main(s) 


5 

March  2000 

Joseph  Morin,  Integrated  System  Diagnostics,  Inc.  (ISD) 

Al  Pfiugrad  (pflugrad_al@prc.com) 

www.prc.com 

5500  employees,  2500  are  within  the  scope  of  SW-CMM 
efforts 

6  people  per  project,  where  the  typical  project  is  a  sin¬ 
gle-year  or  annualized  task  order. 

50-200  KSLOC/year 

Litton/PRC  spans  two  major  domains: 

Software  and  Services  for  National  Defense  Systems 
Software  and  Services  for  Civil  Government  Systems 


In  March  2000,  PRC  received  a  PRC-wide  level  5  rating  in  which  the  assessment 
team  rated  at  the  practice  level  for  all  process  areas  In  the  SW-CMM  v1 .1  model. 
This  major  milestone  is  the  last  of  many.  PRC  initiated  model-based  process  im¬ 
provement  in  1993.  PRC  sites  attained  level  2  in  December  1995  and  PRC  sectors 
achieved  level  3  in  June  1996.  PRC  developed  a  combined  SE/SW  model-based 
program  in  June  1997  and  secured  its  initial  ISO  9001/9902  registrations  in  May 
1998.  In  addition  to  other  ISO  registrations,  PRC  received  a  PRC-wide  level  3  rating 
in  June  1999. 

PRC’s  integrated  many  different  quality  approaches  into  one  multi-faceted  quality 
infrastructure:  CMM-based  process  improvement,  quality  improvement  (Qualtec 
TQM),  ISO  9000,  Quality  Assurance,  Customer  Satisfaction,  and  Employee  Satisfac¬ 
tion.  The  quality  improvement  facet  contains  the  foundational  principles,  teams,  and 
methods  upon  which  all  other  facets  are  built.  PRC  has  pioneered  the  integration  of 
the  SE-CMM,  SECM,  and  SW-CMM,  and  has  participated  in  the  development  of  the 
CMMI  framework  and  associated  models  and  representations. 

ROI  and  Improvement  Trend  Data 

PRC’s  budget  for  engineering  process  improvement  has  exceeded  $1 M  per  year 
since  1 993.  This  figure  is  supplemented  by  various  line  expenditures.  The  following 
characterize  the  business  benefit  PRC  has  received  since  its  first  process  capability 
baseline  (September  1 999): 

•  Defects  in  delivered  documentation  are  down  78%. 

•  Defects  in  delivered  code  are  down  70%. 

•  Defects  found  operationally  are  down  60%. 

•  PRC’s  ability  to  estimate  costs  on  a  monthly  basis  has  increased  32%. 

•  PRC’s  ability  to  meet  monthly  cost  goals  increased  by  40%  (CPIm  is 
.977;  1 .000  is  where  planned  monthly  costs  equal  actual  monthly  costs). 

•  PRC’s  ability  to  meet  monthly  schedule  goals  increased  by  7%  (SPIm  is 
.980;  1.000  is  where  planned  monthly  schedule  equals  actual  monthly 
schedule). 

Barriers  to  Achieving  High  Maturity 

To  achieve  level  4,  PRC  had  to  overcome  several  barriers. 
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First,  PRC  needed  to  resist  applying  level  4  only  to  software- related  activities.  In¬ 
stead,  we  adopted  the  level  4  requirements  to  the  broader  business  issues  of  profit¬ 
ability  and  business  development  based  on  past  and  present  performance.  These 
business  issues  transcended  software  development  and  yet  still  could  be  applied  to 
it.  PRC  worked  to  select  the  few  goals  and  measures  that  were  most  meaningful  to 
all  projects  and  that  had  a  sufficient  stream  of  continuous  data. 

Secondly,  PRC  needed  to  resist  applying  only  organizationally  mandated  goals  and 
measures  on  projects.  Through  pre-assessment  consultation,  PRC  realized  that 
quantitative  management  should  be  applied  to  a  project’s  "points  of  pain."  When 
projects  discovered  that  quantitative  management  could  address  the  very  real  prob¬ 
lems  they  were  facing,  resistance  to  implementation  changed  to  enthusiasm. 

Thirdly,  PRC  needed  to  think  quantitatively  -  to  value  quantitative  management  and 
to  see  applications  of  it  to  existing  problems.  While  this  ability  comes  with  practice,  it 
was  difficult  to  envision  the  end  result  during  initial  implementation  of  level  4  princi¬ 
ples. 

Unique  or  Distinguishing  Practices 

Some  distinguishing  practices  of  high  maturity  organizations  include:  performing 
process  improvement  for  business  reasons,  not  just  process  maturity  goals;  manag¬ 
ing  by  fact;  respecting  people;  applying  process  improvement  principles  to  non¬ 
model  areas;  reducing  and  simplifying  processes  and  process  assets  for  widespread 
use;  and  leveraging  corporate  infrastructure,  past  improvements,  and  best  practices. 

People  and  Cultural  Issues 

On  one  hand,  PRC  has  historically  maintained  that  the  principles  of  quality  organiza¬ 
tions  can  be  applied  regardless  of  the  organizational  process  maturity.  That  is  why 
PRC  implements  respect  for  people,  managing  by  fact,  continuous  improvement, 
and  customer  satisfaction  as  foundational  principles  for  all  projects  and  teams. 

On  the  other  hand,  as  PRC’s  process  improvement  program  has  matured,  it  has  had 
to  maintain  momentum  and  move  from  a  program  targeted  to  innovators  to  one  tar¬ 
geted  at  the  majority  of  managers  and  staff.  Process  improvement  personnel  are 
now  assigned  to  various  levels  of  management,  much  as  contract  and  HR  personnel 
are. 

Continuing  Improvement 

PRC  is  actively  pursuing  improvements  to  increase  project  performance  within  its 
major  business  areas.  First,  PRC  is  implementing  widespread  use  of  quantitative 
management  within  all  organizational  units.  PRC  management  has  begun  rollout  and 
review  of  PRC-wide  quantitative  project  management  initiatives  for  given  types  of 
projects  and  values  during  monthly  operational  reviews.  Secondly,  PRC  is  adding 
processes  and  process  improvement  support  for  non-model  process  areas  like  in¬ 
formation  assurance,  COTS  integration,  network  management,  transition  planning, 
database  administration,  etc.  PRC  believes  that  the  CMMI  is  a  process  framework 
flexible  enough  to  add  support  for  these  engineering  processes,  and  therefore,  PRC 
is  actively  transitioning  to  full  CMMI  implementation.  Thirdly,  PRC  is  pre-tailoring 
corporate  processes  to  program  units  to  reduce  or  eliminate  the  amount  of  tailoring 
necessary  at  the  task  order  or  small  project  level  within  our  business  domains.  Fi¬ 
nally,  PRC  is  refining  its  internal  assessment  methods  to  include:  1)  targeted  as- 
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sessments  using  a  subset  of  CMMI  process  areas  within  the  continuous  representa¬ 
tion,  and  2)  informal  interim  assessments  based  upon  periodic  QA  process  audits. 

Summary 

To  quote  Winston  Churchill,  level  5  is  not  the  end;  it  is  not  even  the  beginning  of  the 
end;  but  it  may  be  the  end  of  the  beginning.  Level  5  gives  an  organization  the  toois  it 
needs  to  independentiy  and  continuously  address  and  resolve  its  own  business  is¬ 
sues.  Specificaiiy,  level  5  gives  PRC  the  ability  to  manage  by  fact,  to  quantitatively 
increase  performance,  and  to  provide  process  power  to  each  employee. 
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This  work  is  sponsored  by  the  National  Applied  Software  Engineering  Center,  which  is  oper¬ 
ated  by  CTC  and  funded  by  the  Defense  Advanced  Research  Projects  Agency.  The  content  of 
the  information  does  not  necessarily  reflect  the  position  or  the  policy  of  the  government,  and 
no  official  endorsement  should  be  inferred. 

INTRODUCTION 

This  paper  provides  a  brief  background  of  CTC’s  extensive  company-wide  quality  initiatives. 
These  initiatives  were  based  on  the  International  Organization  for  Standardization  (ISO)  9001 
Quality  and  the  ISO  14001  Environmental  Management  Systems  standards  and  the  NSD’s 
previous  initiatives  to  achieve  Maturity  Level  2  (L2)  of  the  Capability  Maturity  Model  for 
Software  (SW-CMM).  The  Division’s  current  transition  to  the  Capability  Maturity  Model 
Integrated  for  Systems  Engineering/Software  Engineering  (CMMI-SE/SW)  is  discussed  in 
detail.  The  Division’s  diverse  project  and  product  types  present  significant  challenges  for 
transition  to  and  implementation  of  the  CMMI-SE/SW.  NSD’s  approaches  to  application  of 
the  model  in  a  Research  and  Development  (R&D)  environment,  to  include  tailoring  guide¬ 
lines,  configuration  management  challenges,  quantitative  measures  and  quality  control  should 
be  of  particular  interest.  The  paper  includes  discussions  of  the  Division’s  approach  to  institu¬ 
tionalizing  the  use  of  CMMI-SE/SW  principles  and  the  methods  used  to  strive  for  organiza¬ 
tion-wide  participation. 

Background  -  CTC’s  Quality  Initiatives 

ISO  9001/14001 

CTC  manages  programs  and  performs  technical  services  that  are  critical  to  national  competi¬ 
tiveness  and  defense  objectives.  This  work  requires  quality  products  and  services  across  a 
wide  variety  of  domains.  In  August  1998,  CTC  became  the  Nation’s  first  not-for-profit  com¬ 
pany  to  be  concurrently  registered  to  ISO  9001  and  ISO  14001  Quality  Standards. 

Numerous  procedures  were  developed  and  implemented,  ensuring  that  quality  is  built  into  all 
phases  of  CTC  products  and  services,  including  Quality  Planning,  Design  Review,  Project 
Planning,  Product  Development  and  Delivery,  and  Project  Review.  CTCs  internal  Web  site 
provides  employees  easy  access  to  this  information,  in  addition  to  CTC  Quality  Management 
System  (QMS)  newsletters,  audit  results,  and  QMS/Environmental  Management  System 
(EMS)  training  presentations. 

Capability  Maturity  Model  for  Software  Implementation 

In  1998,  CTC’s  National  Security  Division  embarked  on  an  effort  to  apply  SW-CMM  princi¬ 
ples  to  software  projects  under  the  Division’s  auspices.  This  effort  focused  on  developing 
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SW-CMM  compliant  NSD  processes  within  the  existing  CTC  ISO  9001/14001  QMS/EMS 
framework. 

NSD  used  an  organization-wide  approach  to  develop  and  implement  SW-CMM  principles. 
The  effort  culminated  in  a  CMM  Based  Assessment  for  Internal  Process  Improvement  (CBA- 
IPI)  that  assessed  the  NSD  at  SW-CMM,  Version  1.1  Maturity  Level  2  in  September  1999. 
After  the  CBA-IPI,  the  NSD  initiated  an  action  plan  to  progress  to  SW-CMM  Maturity  Level 
3  (L3). 

TRANSITION  TO  CMMI-SE/SW 

The  majority  of  the  NSD’s  programs  are  Department  of  Defense  (DOD)  related.  The  pro¬ 
grams  are  highly  diverse,  ranging  in  nature  from  Service  projects  (trade  studies,  consulting, 
etc)  to  Systems  Development  projects  (projects  that  involve  hardware/software  develop¬ 
ment). 

In  March  2000,  the  Software  Engineering  Process  Group  (SEPG)  recommended  a  transition 
to  the  CMMI-SE/SW  based  on  the  model  being  more  applicable  to  the  diverse  NSD  pro¬ 
grams.  NSD  Senior  Management  agreed,  and  established  a  Transition  Team  that  included 
four  members.  Three  members  had  SW-CMM  L2  experience  and  one  had  SW-CMM  L3  ex¬ 
perience,  in  addition  to  experience  adapting  SW-CMM  L3  Key  Process  Areas  (KPAs)  to  in¬ 
clude  Systems  Engineering.  The  team  focused  on  developing  policy  and  procedures.  The 
team  also  functioned  as  the  SEPG  action  arm.  The  Software  Engineering  Process  Group 
name  was  changed  to  Systems  Engineering  Process  Group  to  emphasize  the  broader  scope  of 
systems  engineering. 

Using  the  Software  Engineering  Institute’s  (SEI)  mapping  example,  team  members  mapped 
the  CMMI-SE/SW  L2  and  L3  Specific/Sub  Practices  to  existing  CTC  ISO  9001/140001  Pro¬ 
cedures,  Work  Instructions,  and  to  the  existing  NSD  SW-CMM  L2  processes  and  procedures. 
Regular  weekly  mapping  meetings  were  used  to  study  and  discuss  specific  Process  Areas 
(PAs),  which  led  to  a  gap  analysis.  The  team  built  a  spreadsheet  documenting  500  issues  re¬ 
quiring  modifications  to  existing  procedures  and  creation  of  new  procedures. 

Project  Types  and  Diversity 

The  Division’s  diverse  programs  presented  a  challenge  to  develop  a  standard  process  with 
effective  tailoring  guidelines.  Working  with  the  SEPG,  Senior  Management  made  key  deci¬ 
sions  regarding  the  tailoring  of  the  new  model  to  our  projects.  The  NSD  General  Manager 
decided  early  on  that  Service  programs  would  only  follow  the  procedures  available  in  CTC's 
ISO  9001/14001  arsenal. 

Ultimately,  we  defined  NSD  Programs  as  falling  into  two  project  types,  one  of  which  has 
three  categories. 

■  Service:  Follows  CTC  ISO  9001/14001  compliant  process/procedures 

■  Systems  Development:  Follows  CTC  ISO  9001/14001  procedures  and  NSD  CMMI- 
SE/SW  compliant  processes  and  procedures 

-  Basic  Research  and  Development  (R&D) 

-  Applied  R(S:D/Technology  Demonstration 

-  Applications  Engineering/Product  Development 


46 


CMU/SEI-2001-TR-015 


Tailoring  Guidelines  for  Systems  Development  Projects 

All  NSD  Systems  Development  programs  are  tailorable.  The  NSD  Standard  Process  de¬ 
scribes  how  a  program  can  apply  tailoring  guidelines.  The  guidelines  correspond  to  the  pro¬ 
ject’s  category,  size  and  other  thresholds  as  defined  in  supporting  documents. 

The  project  stages  (conceptual,  preliminary,  critical,  and  final  stage)  also  affect  tailoring 
guidelines.  During  the  Conceptual  Design  Stage,  a  Project  Plan  is  required.  This  plan  in¬ 
cludes  all  required  areas,  some  of  which  may  be  discussed  at  the  macro  level.  The  areas  may 
evolve  into  separate  plans  as  projects  progress  through  the  stages. 

Project-specific  tailoring  guidelines  are  implemented  based  on  the  Project  Manager’s  direc¬ 
tion  and  NSD  Quality  Assurance  Lead  consultation.  Managers  tailor  plans  to  meet  specific 
needs,  eliminating  a  percentage  of  a  procedure  without  requesting  permission.  If  a  threshold 
is  exceeded,  the  SEPG  must  review  and  approve  the  proposed  tailoring. 

Plan  Completion  Timeline  Examples 

A  project  that  accomplishes  Basic  R&D  could  remain  in  the  Conceptual  Design  Stage,  con¬ 
stantly  generating  alternative  solutions.  This  project  usually  has  broadly  defined  requirements 
and  no  deliverables  other  than  a  paper  or  a  rough  breadboard. 

An  Applied  R&D/Technology  Demonstration  project  may  cycle  between  the  Conceptual  and 
Preliminary  Design  Stages.  Generally,  requirements  are  loosely  stated  at  a  macro  level.  The 
stakeholder  set  is  generally  larger  than  that  of  Basic  R&D  and  use  of  the  end  product  is  typi¬ 
cally  more  formal. 

An  Applications  Eneineering/Product  Development  project  generally  has  better  defined  re¬ 
quirements  than  the  two  R&D  project  categories.  End  products  may  be  a  finalized  require¬ 
ments  set,  a  prototype  or  demonstration  system,  or  a  final  end  product. 

Tailoring  Disagreement  Handling  Process 

The  Project  Manager  and  QA  Lead  discuss  proposed  tailoring.  If  disagreement  about  the  tai¬ 
lored  process,  procedure,  or  plan  remains,  it  is  referred  to  the  SEPG  for  review.  If  consensus 
cannot  be  reached  through  the  SEPG,  the  General  Manager  makes  the  final  decision. 

Configuration  Management 
Rigid  Flexibility 

The  transition  team  recognized  the  necessity  of  “rigidly  flexible”  plans.  This  approach  to 
Configuration  Management  (CM)  was  due  to  our  diverse  “product  lines.”  Major  programs 
often  have  multiple  tasks,  which  are  aligned  as  subordinate  projects.  In  this  case,  CM  can  be 
handled  consistently  for  all  sub-projects  or  specifically  for  each  sub-project. 

Web  Configuration  Management  Considerations 

Web-based  projects  typically  involve  database,  web  architecture,  and  interface  or  presenta¬ 
tion  data.  The  presentation  data  can  change  every  two  to  six  weeks.  Managing  rapid  life  cycle 
change  demanded  a  new  approach.  In  this  case,  only  macro  components  may  be  under  CM, 
such  as  the  database  and  architecture.  Major  components  changes  must  undergo  a  Configura¬ 
tion  Control  Board  (CCB)  process.  “Presentation  data”  is  managed  as  regular  data.  Depend¬ 
ing  on  the  criticality  of  the  content  data,  it  is  backed  up  weekly,  daily,  or  even  hourly. 

The  client  often  calls  for  rapid  changes  to  the  presentation  data.  Changes  are  handled  as  data 
modifications.  The  call  is  documented  on  a  standardized  form.  The  Project  Manager  and  the 
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client  review  changes.  Two  servers,  “staging”  and  “live”  are  in  use.  Modified  presentation 
data  are  placed  on  the  staging  web  server  for  testing  and  access  is  given  to  the  stakeholders. 

Team  members  test  modifications  to  presentation  data  on  the  staged  server,  ensuring  changes 
do  not  affect  “live”  data.  Once  stakeholder  feedback  is  obtained,  the  presentation  data  is  mi¬ 
grated  to  the  live  web  server.  When  no  feedback  is  available,  the  Project  Manager  can  ap¬ 
prove  data  transfer  to  the  “live”  web  server. 

Configuration  Management  of  Multiple  Inputs 

Another  example  project  deals  with  coordinating  the  activities  of  our  personnel  with  the  work 
and  activities  of  other  organizations.  CTC  personnel  are  responsible  for  validating  that  candi¬ 
date  specifications  and  emerging  R&D  products  conform  to  accredited  standards  (IEEE,  ISO, 
etc.).  This  is  accomplished  by  running  tests  against  the  specifications  and  products  to  validate 
conformance. 

Team  members  send  post-test  feedback  to  the  originating  organizations  (Academia,  Consor¬ 
tia,  and  Private  Companies).  CM  is  accomplished  on  the  test  results.  Only  when  all  stake¬ 
holders  agree  are  these  candidate  products  accepted  in  the  accredited  standard.  When  this 
occurs,  the  accrediting  body  provides  CM. 

Measures  and  Analysis 

The  new  Measures  and  Analysis  PA  caused  us  to  look  hard  at  how  we  capture  level  of  effort 
(labor  hours)  in  our  projects.  We  developed  a  Process  Improvement  Database,  which  collects 
12  core  project  activities.  This  provides  organization-wide  indicators  of  systems  development 
processes  effectiveness.  This  database  is  linked  to  the  company’s  electronic  timesheet  system 
to  provide  automated  collection  of  NSD  labor  data. 

All  NSD  programs  have  a  basic  job  number.  Major  tasks  within  a  program  are  identified  by 
subordinate  job  numbers.  Each  subordinate  job  number  has  a  suffix  that  is  based  on  12  core 
measures  in  which  NSD  senior  management  is  interested.  As  each  individual  completes  work 
on  a  project,  he  or  she  uses  the  unique  project  number  and  suffix  that  identifies  the  accom¬ 
plished  work. 

Quality  Assurance 

NSD  Quality  Assurance  operates  within  the  CTC  ISO  auditing  system.  The  NSD  QA  Lead, 
who  is  also  an  Internal  ISO  Auditor  or  an  ISO  Auditor  evaluates  NSD  programs  based  on 
approved  plans,  waivers,  and  tailoring. 

We  created  a  Maturity  Database  of  CMMI-SE/SW  assessment  questions.  The  NSD  QA  Lead 
uses  this  database  during  audits.  The  NSD  QA  Lead  can  insure  randomized  question  selec¬ 
tion  is  obtained.  Results  are  placed  in  an  Assessment  Database,  which  facilitates  tracking  of 
audit  results  over  time. 

CrC’s  Approach  to  Institutionalizing  CMMI-SE/SW 

The  company-wide  training  for  Program  Managers  and  Project  Managers  is  based  on  the  Pro¬ 
gram  Management  Institute’s  Body  of  Knowledge  (PMBOK™),  the  CTC  ISO  9001/140001 
Quality  Policy,  Procedures,  and  Work  Instructions.  Training  for  the  new  model  required 
change  management. 

Early  on,  we  recognized  that  a  different  approach  was  needed  to  inculcate  the  processes  and 
practices  of  the  model  into  our  daily  operations.  Our  previous  SW-CMM  transition  training 
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emphasized  the  vocabulary  of  the  model.  Instead,  we  packaged  the  new  NSD  Systems  De¬ 
velopment  Policy,  Standard  Process  and  Procedures  in  our  familiar  NSD  vocabulary. 

The  new  CMMI-SE/SW  PAs  and  their  specific  practices  are  mapped  to  the  new  documents  to 
ensure  full  coverage  of  all  the  PAs.  However,  only  the  NSD  Policy  Statement  and  Standard 
Process  documents  contain  direct  references  to  the  CMMTSE/SW.  No  mention  is  made  of 
the  model  in  our  new  Procedures,  Work  Instructions,  or  Templates. 

Incorporation  of  the  CMMI-SE/SW  principals  within  the  CTC  QMS/EMS  and  the 
PMBOK™  framework  permits  us  to  train  the  new  systems  development  process  rather  than 
the  model.  This  methodology  lessens  potential  resistance  to  change,  as  well  as  keeping  the 
staff  from  viewing  the  new  processes  as  “an  extra  burden.”  Instead,  training  focuses  on  the 
CTC  and  NSD  way  of  doing  business,  with  the  CMMI-SE/SW  silently  embedded  in  the 
Processes,  Procedures,  and  Work  Instructions. 

SUSTAINING  MOVEMENT  TOWARD  CMMI-SE/SW  MATURITY  LEVEL  3  AND 
BEYOND 


Early  in  the  transition  to  CMMI-SE/SW  a  briefing  for  Senior  Management  covered  the  modi¬ 
fied  business  procedures  that  would  be  required  to  incorporate  L2  and  L3  features  of  the 
model.  Numerous  review  levels  and  multiple  reviews  were  used  to  insure  middle  and  senior 
management  buy-in  to  the  new  process  and  procedures: 


Process  Improvement  Team; 
SEPG  Review; 

Mid-level  Managers: 

Senior  Management  Review: 
General  Manager  Approval; 


Initial  Reviews 

Several  review  cycles 

Reviews  during  Bi-weekly  meetings 

Draft  Procedure  Review 

Final  Approval 


■  The  GM  issues  the  Systems  Development  Policy  and  Standard  Process. 

■  The  Chairman  of  the  SEPG  is  the  approval  authority  for  the  NSD  Systems  Development 
Procedures. 

■  Senior  Managers,  Program  Managers,  and  Project  Managers  participate  in  bi-weekly 
meetings,  which  serve  as  a  vehicle  for  discussions  and  feedback. 

■  The  team  develops  NSD  Systems  Development  Process  Overview  briefing.  Procedures 
training,  and  Project  Plan  training. 

■  Transition  team  members  serve  as  mentors  to  specific  programs  to  ensure  continuity  in 
assistance. 

While  only  applicable  to  NSD  projects,  the  policy,  process,  procedures  and  supporting  docu¬ 
ments  are  available  to  the  entire  company  workforce  on  the  Intranet  and  can  be  used  by  any 
project. 

Summary 

This  paper  provides  an  overview  of  CTC’s  approach  to  transition  to  the  CMMI-SE/SW.  We 
recognize  that  the  transition  will  continue  to  be  a  challenging  and  dynamic  process  given  the 
diversity  of  projects  addressed  by  CTC.  The  NSD’s  continuing  pursuit  of  process  improve¬ 
ment  is  a  further  endorsement  of  CTCs  commitment  to  achieving  total  client  satisfaction. 
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Introduction 

Although  some  of  the  assessment  teams  in  the  CMMI  pilots  have  experienced  long 
workdays,  such  days  are  not  integral  to  the  CMMI-based  SCAMPI  method.  Long 
days  have  become  typical  of  CMM-based  assessments,  including  CBA  and  EIA  731. 
SCAMPI  assessments  have  followed  this  pattern.  On  the  other  hand,  many  assess¬ 
ment  team  leaders  have  led  assessments  with  very  reasonable  length  days. 

At  TRW,  we  have  found  that  long  days  are  the  result  of  poor  implementation  of  the 
assessment  methods,  and  not  inherent  in  the  methods  themselves.  Maintaining  a  rea¬ 
sonable  assessment  workday  relies  on  several  factors: 

•  Identifying  an  appropriate  number  of  days  for  the  assessment,  based  on  the  scope 
of  the  model  and  organization; 

•  Effective  team  dynamics,  experience,  and  CMM/CMMI  knowledge; 

•  Effectively  using  questionnaire  data  and  document  review  to  focus  the  interviews; 

•  Continual  team  emphasis  on  working  efficiently,  especially  in  data  consolidation 
and  findings  development. 

Assessments  often  suffer  from  four  problems: 

•  The  team  ignores  the  questionnaire  responses; 

•  The  assessed  organization  supplies  poor  quality  evidence  material; 

•  The  team  spends  excessive  time  drafting  "perfect"  words  for  the  findings; 

•  The  team  spends  excessive  time  arguing  about  issues  unrelated  to  the  final  results. 

TRW  experience  indicates  that  typical  assessment  times  could  be  shortened  by  25%- 
50%  by  effectively  using  the  principles  discussed  in  this  paper  to  eliminate  the  scrap 
and  re-work  inherent  in  some  assessments.  This  has  been  our  experience  in  TRW 
CBA  IPI  assessments,  where  the  typical  assessment  day  does  not  exceed  ten  hours, 
the  assessment  is  typically  completed  in  1-2  weeks,  and  the  findings  are  detailed  and 
accurate.  With  the  improvements  in  SCAMPI  over  CBA  IPI,  (e.g,  changing  the  ques¬ 
tionnaire  and  collaboration  rules  to  reduce  assessment  time),  it  should  be  possible  to 
do  cost-effective  assessments. 

Discussion 

TRW  has  used  the  following  mechanisms  to  reduce  cost  and  time  of  assessments, 
while  retaining  high  accuracy  and  repeatability: 
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•  Use  the  questionnaire  responses  to  focus  the  team. 

Use  an  expanded  questionnaire  (like  EIA  731  and  SCAMPI)  that  goes  to  the  prac¬ 
tices  level.  Give  the  organization  a  few  weeks  to  understand  and  complete  the 
questionnaire.  Identify  evidence  in  the  questionnaire.  Use  the  interviews  and 
document  review  to  confirm  the  questionnaire  responses. 

•  Poor  quality  evidence  can  be  improved. 

Another  use  of  the  expanded  questionnaire  can  be  to  monitor  the  progress  of  pro¬ 
jects  to  ensure  that  they  are  making  progress  in  the  proper  identification  of  evi¬ 
dence.  TRW  ensures  that  the  projects  understand  what  they  are  doing  to  support 
the  assessment  by  monthly  reviews.  This  method  also  simplifies  the  process  of 
documentation  review  for  the  KPA  mini-teams.  Using  this  approach  usually  di¬ 
minishes  the  need  for  documentation  requests  and  almost  always  shortens  the 
need  for  multiple  days  of  documentation  reviews. 

•  Don’t  write  “random”  observations,  simply  rate  each  practice. 

“Practice  is  satisfied  through _ ”  or  “Practice  is  not  satisfied  because _ ”. 

Trust  the  KPA  mini-teams  to  compile  preliminary  findings.  During  the  on-site 
team  training  generate  example  findings  to  guide  the  mini-teams.  Use  standard 
phrases  related  to  practice  satisfaction.  Review  and  wordsmith  findings  as  a  team. 

Other  Efficient  Practices 

The  following  additional  practices  should  be  considered: 

•  Don’t  use  a  formal  assessment  when  an  informal  assessment  is  more  appro¬ 
priate. 

Unless  a  rating  is  required  you  should  consider  performing  an  informal  assess¬ 
ment.  Not  only  is  this  less  of  an  impact  on  the  organization  but  it  also  is  far  less 
costly.  When  performed  like  a  CAP  compliant/  SCAMPI/  or  ARC  it  should  pro¬ 
vide  sufficient  findings  to  provide  the  organization  with  guidance  for  the  genera¬ 
tion  of  a  process  improvement  plan.  It  will  also  provide  the  projects’  and  organ¬ 
izational  interviewees  some  experience  in  the  assessment  process. 

•  Don’t  schedule  long  days! 

Plan  to  perform  the  assessment  like  it  is  a  project.  Don’t  schedule  it  so  you  have 
no  chance  to  recover.  Work  both  efficiently  and  effectively  and  the  team  will 
strive  to  make  the  days  productive.  If  you  plan  on  “long”  days  the  team  will  not 
feel  any  constraints  to  achieve  the  goals  in  a  normal  (8:00  AM  to  6:00  PM)  day. 
The  only  possible  long  day  should  be  Wednesday  when  you  will  generate  the 
Draft  Findings  but  that  should  be  the  only  one. 

•  Start  with  an  overview  of  the  organization’s  process  and  documentation. 

Most  of  the  team  members  need  an  introduction  to  the  organization  and  to  the 
programs.  Have  the  SEPG  prepare  a  site  briefing  that  follows  a  template  with  the 
basic  material  requested  in  the  “process  appraisal  information”  -  e.g.,  organiza¬ 
tion  and  project  questionnaires,  project  and  organization  charts.  They  should  also 
brief  a  description  of  the  organization’s  processes  and  process  structure.  This  fur¬ 
ther  supports  enabling  the  assessment  team  to  enter  into  the  interviews  with  a 
crisp  understanding  of  the  organization  being  assessed. 
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•  Have  the  organization  compile  all  of  the  evidence  ahead  of  time. 

As  stated  above  have  the  projects  generate  their  answers  to  the  expanded  ques¬ 
tionnaire  and  gather  those  artifacts.  These  artifacts  can  be  placed  into  folders  (in 
CMMI  Process  Area  and  practice  order)  for  the  assessment. 

•  Don’t  review  documents  until  after  the  interviews. 

By  having  the  site  briefing  and  descriptions  of  the  projects  and  organization  it  is 
reasonably  easy  to  enter  into  the  interviews.  With  only  a  minimal  review  of  the 
evidence  materials  performed  by  the  KPA  mini-teams  questions  can  be  formu¬ 
lated  to  evaluate  the  interviewees  on  the  performance  of  the  activities  required  by 
SCAMPI.  Once  the  interviews  are  completed  it  should  be  easy  to  validate  any 
discrepancies  each  day  (or  after  an  interview  session)  to  see  if  any  follow  up  in¬ 
terviews  are  needed.  This  also  facilitates  generation  of  initial  findings  to  maxi¬ 
mize  the  effectiveness  of  the  team. 

•  Don’t  use  more  that  50%  inexperienced  team  members. 

Experience  is  the  single  biggest  factor  in  the  efficiency  of  doing  assessments.  We 
strike  a  balance  between  the  need  to  train  new  assessors,  and  efficiency.  By  lim¬ 
ited  new  assessors  to  one  per  mini-team,  experienced  team  members  can  provide 
the  mentoring  needed  to  support  both  efficient  and  effective  team  activities.  If 
you  have  an  imbalance,  the  Lead  Assessor  or  one  of  the  more  experienced  team 
members  must  ensure  the  teams  get  the  proper  support  needed  to  learn  the  process 
and  to  generate  good  findings.  In  addition  you  should  always  designate  and  train 
at  least  one  alternate,  who  can  take  the  place  of  a  team  member  who  is  unable  to 
participate  at  the  last  minute  due  to  health  or  personal  emergency. 

•  Use  tools  to  gather  questionnaire  data  and  consolidate/review  findings. 

Tools  can  especially  be  useful  in  analyzing  and  summarizing  the  expanded  ques¬ 
tionnaire,  and  can  be  used  by  the  organization  to  prepare  for  the  assessment. 

Conclusions 

The  TRW  assessment  practices  provide  an  opportunity  for  other  assessment  teams  to 
improve  their  processes.  Essentially,  we  believe  the  assessment  community  should 
apply  the  CMM  principles  to  assessments:  disciplined  adherence  to  reasonable  plans 
and  continuous  improvement. 
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Integrating  Process  Maturity  Review  into  Project  Management  Reviews 

SuZ  Garcia,  US  Deployments  Manager,  aimware  Inc. 

Position  Paper  for  CMMI  Transition  Workshop 


Introduction 

Organizations  have  been  using  Capability  Maturity  Models  (CMMs)®  as  a  means  of  improving  their 
organizational  and  project  management  practices  for  several  years,  with  significant  positive  results 
achieved  in  many  organizations,  especially  in  the  software  development  area,  where  the  models  were 
first  introduced  in  1991.  As  far  back  as  1994  (Bate  et  al,  1994)  "generic  practices"  for  process  man¬ 
agement  have  been  proposed  that  provide  an  evolutionary  set  of  steps  to  take  in  improving  an  individ¬ 
ual  process,  for  example,  the  risk  management  process  or  the  management  review  process.  Each  evolu¬ 
tionary  step  is  called  a  "level",  and  the  framework  groups  the  generic  practices  into  6  levels,  from  "not 
practiced"  (Level  0)  through  "optimizing"  (Level  5).  As  the  project’s  or  organization’s  practices  are 
identified  as  being  consistent  with  those  at  each  level,  the  process  is  said  to  become  more  capable,  in¬ 
dicating  that  that  process  is  more  likely  than  one  at  a  lower  capability  level  of  achieving  stated  process 
and  product  objectives,  although  factors  other  than  capability  are  acknowledged  to  affect  actual  proc¬ 
ess  performance.  The  most  recent  published  expression  of  these  practices  is  the  CMM  Integration 
Framework  1 .02  (SEI  2000) 

One  potential  use  of  these  practices  is  as  a  project  management  checklist  that  can  be  applied  in  reviews 
and  other  project  oversight  activities  to  help  understand  how  project  management  practices  are  evolv¬ 
ing  during  the  course  of  a  project,  and  to  set  targets  for  the  practices  that  will  be  performed  on  that  or  a 
future  project.  In  addition,  information  from  the  reviews  can  help  project  managers  understand  when 
an  organization  is  NOT  ready  for  additional  practices,  due  to  foundational  elements  identified  as  being 
missing. 

For  example,  at  Capability  Level  2,  the  "Managed"  level,  the  generic  practices  include  such  items  as  " 
Provide  adequate  resources  for  performing  the  planned  process,  developing  the  work  products  and 
providing  the  services  for  the  process,"  (GP2.3)  and  "  Place  designated  work  products  of  the  organiza¬ 
tional  process  focus  process  under  appropriate  levels  of  configuration  management. ."  (GP  2.6)  At 
Capability  Level  3,  the  practices  evolve  to  include  treating  the  process  performed  on  the  project  as  an 
organizational  asset. 

The  sections  below  provide  an  analysis  of  selected  generic  practices  in  terms  of  their  utility  for  use  in 
reviewing  a  project  that  is  underway,  and  provides  rationale  for  the  selection  and  use  of  the  checklist 
during  project  management  reviews. 


The  context  of  project  management  reviews... 

In  my  experience  acting  either  as  a  project  manager  or  as  a  project  participant,  project  management 
reviews  have  been  focused  primarily  on  the  accomplishments  and  challenges  directly  related  to  the 
evolution  of  the  project’s  work  products  from  one  state  to  the  next.  In  this  regard  I’m  speaking  of  the 
types  of  project  management  reviews  that  tend  to  signal  the  transition  of  a  project  from  one  major  life 
cycle  phase  to  another.  Specifically,  reviews  such  as  a  System  Requirements  Review,  Preliminary  or 
Critical  Design  Review,  and  Test  Readiness  Reviews  are  examples  of  the  types  of  reviews  that  are  the 
intended  focus  of  this  paper.  Although  this  focus  on  product  evolution  is  necessary  to  determine  the 
effect  of  decisions  being  made  about  the  project’s  next  set  of  steps,  I  would  argue  that  project  man¬ 
agement  reviews,  by  virtue  of  their  ability  to  bring  the  stakeholders  of  the  project  into  the  same  room 
(either  physically  or,  in  today’s  environment,  virtually),  provide  an  often-missed  opportunity  to  review 
and  influence  the  work  and  management  processes  that  are  being  used  to  move  the  project  forward. 
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The  context  of  process  maturity  reviews/appraisals... 

Many  organizations  use  formal,  periodic  (not  synchronized  with  projects,  usually)  appraisals  of  proc¬ 
ess  maturity  to  gain  confidence  that  their  process  improvement  efforts  are  bearing  fruit.  These  apprais¬ 
als  can  range  from  very  informal  discussions,  to  questionnaire-based  surveys  (Whitney,  1994)  to  more 
rigorous  and  intense  appraisal  methods  that  involve  interviews  of  multiple  levels  of  management  and 
documentation  reviews  (Masters  et  al,  1996).  These  appraisals  typically  involve  multiple  projects 
within  an  organization  or  business  unit.  Although  they  provide  synthesized  data  about  the  tendencies 
in  performance  of  various  practices  on  projects,  generally  they  do  not  provide  specific  information 
about  particular  practices  that  have  been  seen  or  not  seen  in  one  of  the  surveyed  projects.  This  confi¬ 
dentiality  is  intended  to  encourage  project  team  members  to  honestly  disclose  the  practices  or  lack 
thereof  that  are  present  on  the  projects  of  interest.  It  is  believed  that  disclosing  the  exact  practices  of 
each  project  to  other  members  (especially  senior  management)  of  the  organization  could  lead  to  puni¬ 
tive  actions  if  a  project  is  not  meeting  the  expectations  of  the  organization.  In  cultures  who  are  just 
beginning  their  improvement  efforts,  or  who  have  very  high  expectations  and  insufficient  resources  to 
improve  the  project’s  practices,  there  is  a  decided  risk  related  to  revealing  information  specific  to  an 
individual  project. 

However,  I  have  also  encountered  more  than  a  few  organizations  for  whom  the  use  of  such  data  would 
not  be  punitive;  rather,  it  would  be  to  help  improve  the  performance  of  that  project  in  its  current  state 
or  to  improve  projects  that  directly  follow  from  it.  For  these  organizations,  traditional  viewpoints  of 
confidentiality  of  project  data  from  a  process  maturity  appraisal  are  less  useful,  and  even  can  be  per¬ 
ceived  by  project  managers  and  participants  as  a  barrier  to  helping  them  improve  their  local  practices. 

What  are  the  concepts  of  CMMs  that  are  relevant  to 
project  management  review? 

As  part  of  the  Software  CMM  vl.l  author  team,  I  was  struck  at  how  easily  the  topics  in  the  CMM  gen¬ 
erally  fit  into  one  of  three  categories:  technical,  project  support,  or  organizational  support.  We  catego¬ 
rized  the  Key  Process  Areas  (major  topics)  of  the  SW-CMM  according  to  these  categories,  and  when 
we  abstracted  the  process  management  concepts  embedded  in  the  SW-CMM  into  a  set  of  “generic” 
practices  that  were  used  to  seed  the  ISO  15504  standards  effort  (called  the  SPICE  project  at  the  time) 
and  the  Systems  Engineering  CMM,  the  categorization  of  practices  into  these  three  areas  was  very 
helpful  to  the  team  in  understanding  the  evolution  of  behaviors  that  a  CMM  was  intended  to  support. 

To  help  understand  the  application  of  the  individual  practices  to  project  management,  it  is  helpful  to 
understand  the  intent  of  each  capability  level.  I  tend  to  frame  the  general  concepts  of  each  level  in 
terms  of  different  types  of  learning  that  is  being  encouraged  in  the  organization  using  the  practices. 

The  following  paragraphs  summarize  my  perspective  on  how  the  different  capability  levels  reflect 
changes  in  the  depth  and  breadth  of  learning  going  on  in  the  organization. 

There  is  only  one  generic  practice  at  Capability  Level  One,  which  basically  says  that  the  practices  in 
the  specific  topic  areas  of  the  model,  the  Process  Areas,  are  performed.  This  provides  a  basis  for  im¬ 
proving  the  fundamental  practices  related  to  a  specific  topic.  The  practices  of  Capability  Level  2,  the 
Managed  Level,  focus  on  behaviors  that  turn  implicit  knowledge  about  how  the  process  is  performed 
and  managed  to  explicit  knowledge  about  how  the  process  is  performed  and  managed.  I  call  this  the 
transition  from  “individual  learning”  to  “local  learning”.  Knowledge  that  used  to  be  in  the  heads  of 
project  members  is  now  accessible  to  other  members  of  the  local  work  group  or  project  via  recorded 
procedures,  templates,  and  other  descriptions  of  the  practices  and  the  results  of  the  project  using  them. 

The  evolution  to  Capability  Level  3,  the  Defined  level,  involves  the  transition  from  “local  learning” 
into  “organizational  learning”.  Knowledge  that  was  previously  only  readily  available  to  the  local  pro¬ 
ject  team  members  is  now  communicated  throughout  the  organization  via  agreed-upon  process  de¬ 
scription  approaches.  Infrastructure  (such  as  defined  training  and  skill  building  events)  that  is  organ¬ 
izational  in  nature  is  available  and  used  to  encourage  the  qualitative  analysis  and  evolution  of  the 
practices.  Tying  project  practices  explicitly  to  organizational  standards  is  an  accepted  approach  to 
gaining  knowledge  about  the  variations  in  project  practices  across  the  organization. 
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The  evolution  to  Capability  Level  4,  the  Quantitatively  Managed  level,  involves  the  transition  from 
“organizational  learning”  to  “quantitative  learning”.  This  is  not  meant  to  imply  that  measurement  is 
not  used  at  lower  levels  to  gain  insight  into  the  performance  of  project  or  organizational  processes. 
However,  at  levels  prior  to  this,  the  expectation  that  measurement  and  quantitative  data  are  a  primary 
support  to  project  and  process  decision  making  is  not  yet  emphasized.  One  of  the  frequently  debated 
questions  among  authors  of  CMMs  is  whether  or  not  all  processes  used  to  support  product  develop¬ 
ment  are  amenable  to  evolution  to  level  4  and  whether  quantitative  insight  is  necessary  for  all  proc¬ 
esses  to  be  optimally  effective.  Certainly  a  significant  subset  of  product  development  and  management 
processes  would  be  deemed  worthy  of  this  investment  in  organizations  that  are  seriously  pursuing 
widespread  process  improvement. 

The  evolution  to  Capability  Level  5,  the  Optimizing  level  (note  the  change  in  language  for  this  level’s 
title.  It  is  intentional.  The  idea  of  Level  5  is  the  optimization  process  is  a  continual  one  that  does  not 
end  just  because  the  definition  of  the  model  has  ended),  involves  the  transition  from  “quantitative 
learning”  to  “complex  learning”.  When  I  use  the  term  complex  here,  Tm  using  it  in  the  context  of 
complexity  and  chaos  theories  that  are  starting  to  be  applied  to  the  management  environment  (Stacey, 
1996).  These  theories  argue  that  complex  adaptive  learning  is  the  hallmark  of  an  agile,  flexible  organi¬ 
zation  that  is  capable  of  responding  to  rapidly  changing  internal  and  external  stimuli.  The  intent  of 
Optimizing  practices  is  to  encourage  the  behaviors  that  would  likely  lead  to  the  kind  of  agility  de¬ 
scribed  by  management  theorists  applying  complexity  concepts  to  organizations.  It  is  my  view  that  the 
first  four  capability  levels  tend  to  focus  on  “operational”  process  management,  while  the  fifth  level 
switches  the  focus  to  “strategic”  process  management.  Although  a  set  of  practices  has  been  written 
(and  rewritten,  and  rewritten!)  to  support  this  intent,  this  level  is  the  most  difficult  to  describe  in  terms 
of  behaviors  reflected  as  practices. 

How  can  CMMs  be  used  to  help  projects  improve  their 
practices  directly? 

The  following  graphic  has  been  used  to  notionally  describe  some  of  the  evolution  concepts  discussed 
above  related  to  CMMs  (Garcia,  1996). 


Process 

Improvement 


Process 

Control 


Qualitative  Quantitative 


Exhibit  A.  Notionai  view  of  Evoiving  Capabiiity  aiong  Different  Dimensions 
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Notice  that  the  figure  looks  at  two  dimensions  to  describe  the  evolution  of  capability  in  an  organiza¬ 
tion:  the  evolution  from  qualitative  to  quantitative,  and  the  evolution  from  control  to  improvement.  If 
you  think  about  improving  project  management  practices  by  paying  attention  to  process  maturity,  these 
two  dimensions  of  evolution,  combined  with  the  learning  concepts  discussed  in  the  previous  section, 
combine  to  provide  a  powerful  framework  for  analyzing  and  improving  project  practices. 

Operationally,  the  generic  practices  at  Capability  Level  2  are  all  focused  on  and  targeted  towards  or¬ 
ganizational  practices;  there  are  additional  project-focused  practices  at  Levels  3,  4,  and  5  that  build  on 
this  initial  set.  Transforming  the  generic  practices  targeted  to  project  issues  into  a  set  of  project  man¬ 
agement  review  questions  is  a  simple  yet  powerful  way  to  help  focus  the  project’s  management  and 
participants  on  paying  attention  to  the  evolution  of  their  process  at  the  same  time  they  are  paying  atten¬ 
tion  to  the  evolution  of  their  products. 

Exhibit  B  is  a  table  that  transforms  the  CMMI  1.02’s  project-focused  Capability  Level  2  generic  prac¬ 
tices  into  a  set  of  review  questions  to  be  incorporated  into  the  agenda  of  project  management.  I  suggest 
using  a  scale  of  1  (practice  rarely  seen  on  this  project)  to  5  (practice  is  always  performed  on  this  pro¬ 
ject)  as  a  “self-rating”  method  for  the  processes  that  are  of  interest  to  the  project,  particularly  the  man¬ 
agement  processes.  A  different  rating  table  can  be  used  for  each  process  of  interest  (for  instance,  there 
may  be  significant  differences  in  how  many  of  these  practices  are  exhibited  within  the  Project  Plan¬ 
ning  process  vs.  the  Data  Management  process).  Some  of  the  questions  are  direct  inversions  of  the 
practice  into  a  question.  Others  separate  the  practice  into  two  separate  questions,  where  my  experience 
indicates  that  there  is  often  a  different  set  of  behaviors  going  on  related  to  the  different  parts  of  a  prac¬ 
tice.  Whenever  you  see  “the  process”,  you  would  want  to  substitute  the  actual  process  of  interest  for 
the  general  term  (e.g.  “the  project  planning  process”). 


Generic 

Practice 

Number 

Generic  Practice  Text 

Related  Review  Question(s) 

GP  2.2 

Establish  and  maintain  the  re¬ 
quirements  and  objectives,  and 
plans  for  performing  the  process. 

To  what  extent  was  the  use  of  this  process  for  this 
project  planned? 

How  often  are  the  plans  for  performing  the  proc¬ 
ess  updated?  (not  a  1-5  question) 

GP2.3 

Provide  adequate  resources  for 
performing  the  planned  process, 
developing  the  work  products  and 
providing  the  services  for  the 
process. 

To  what  extent  are  resources  explicitly  allocated 
for  performing  the  process  and  developing  its 
work  products? 

To  what  extent  are  the  resources  allocated  for 
performing  the  process  considered  adequate? 

GP2.4 

Assign  responsibility  and  author¬ 
ity  for  performing  the  process, 
developing  the  work  products, 
and  providing  the  services  of  the 

Are  responsibility  and  authority  for  developing 
the  work  products  and  providing  the  services  of 
the  process  assigned? 

GP2.5 

process. 

Train  the  people  performing  or 
supporting  the  process  as  needed. 

To  what  extent  are  individuals  performing  the 
process  appropriately  trained  to  support  their  per¬ 
formance  of  the  process? 

GP2.6 

Place  designated  work  products 
of  the  process  under  appropriate 
levels  of  configuration  manage¬ 
ment. 

Are  work  products  important  to  performing  the 
process  identified? 

Are  identified  work  products  of  the  process 
placed  under  version  or  configuration  control? 

GP  2.7 

Identify  and  involve  the  relevant 
stakeholders  of  the  process  as 
planned. 

To  what  extent  are  the  relevant  stakeholders  for 
this  process  identified? 

To  what  extent  are  the  relevant  stakeholders  for 
this  process  involved? 

GP  2.8 

Monitor  and  control  the  process 

At  what  intervals  is  information  about  the  pro- 
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against  the  plan  and  take  appro-  gress  of  this  process  within  the  project  tracked 

priate  corrective  action.  and  made  available?  (not  a  1-5  question) 


GP  2.9  Objectively  evaluate  adherence  of 

the  process  and  the  work  prod¬ 
ucts  and  services  of  the  process  to 
the  applicable  requirements,  ob¬ 
jectives,  and  standards,  and  ad¬ 
dress  noncompliance. 

GP  2.10  Review  the  activities,  status,  and 
results  of  the  process  with  man¬ 
agement  and  resolve  issues. 


To  what  extent  is  appropriate  action  taken  when 
significant  deviations  to  the  plan  for  this  process 
are  identified? 

Are  work  products  verified  against  applicable 
requirements?  (these  could  include  both  product 
and  process  requirements) 

To  what  extent  are  identified  noncompliance  is¬ 
sues  addressed? 

To  what  extent  is  the  status  of  the  process  re¬ 
viewed  with  management?  (this  one  should  be  a 
“gimme”  if  you’re  implementing  this  project 
management  review  approach ! ! !) 


Exhibit  B.  Tabie  of  project-focused  Capabiiity  Levei  2  Generic  Practices  from  CMMi 
1.02 


Since  CMMs  are  additive  in  terms  of  the  expectations  of  the  practices  being  performed,  you  may  find 
it  useful  to  restrict  your  initial  set  of  questions  to  ones  like  those  in  the  table,  without  regard  (initially) 
to  higher  maturity  practices.  Once  at  least  half  of  the  questions  would  be  at  least  at  a  3  on  your  sliding 
scale  (practices  are  performed  frequently)  you  may  want  to  add  the  level  3  practices  that  relate  well  to 
project  management  reviews. 

This  initial  set  of  “translated”  practices  is  only  a  starting  point  for  projects  that  wish  to  have  insight 
into  the  evolution  of  their  project’s  management  practices  using  a  CMM.  Within  CMMI  1.02’s  de¬ 
scription  of  generic  practices  an  elaboration  statement  is  presented  that  provides  information  about 
how  that  generic  practice  might  be  applied  to  an  individual  process  area.  Especially  for  organizations 
that  look  at  one  of  these  practices  and  ask  “What  is  the  scope  of  this  generic  practice?”,  the  elabora¬ 
tions  provide  some  suggestion  of  the  types  of  activities  that  would  be  expected  for  an  individual  proc¬ 
ess  to  demonstrate  use  of  the  generic  practices. 

How  would  project  management  reviews  change  if 
process  maturity  were  inciuded? 

When  I  think  back  to  some  of  the  more  challenging  product  developments  I  have  been  involved  with,  I 
wonder  about  the  effects  that  using  these  questions  or  a  subset  of  these  questions  would  have  had  on 
our  management’s  focus  regarding  the  project’s  processes  and  products.  How  surprised  would  they 
have  been  when  the  answer  to  one  or  more  of  these  questions  didn’t  meet  their  expectations?  How 
would  they  have  reacted?  I  know  some  of  them  would  have  put  some  serious  effort  into  correcting 
process  problems  that  would  be  likely  to  add  project  risk. . ..I  don’t  know  a  single  project  manager  who 
would  investigate  why  work  products  were  "not  often"  put  under  version  control,  for  example!  I  sus¬ 
pect  the  best  of  the  project  managers  I’ve  worked  with  already  had  a  mental  “process  checklist”  that 
they  may  have  used  to  do  their  private  risk  analyses  on  the  project.  As  more  projects  start  looking  into 
these  areas,  I  look  forward  to  future  papers  providing  insight  into  risk  reduction  and  improvements  in 
practices  and  products  that  result  from  this  expanded  focus  for  a  project  management  review. 

Summary 

This  paper  has  presented  a  use  for  generic  practices  of  CMMI  1.02  that  suggests  that  they  can  be 
adapted  into  sets  of  questions  that  can  open  up  the  context  of  project  management  reviews  to  include 
local  process  maturity  review.  CMMI  1.02  contains  practices  that  are  specifically  geared  toward  the 
project  context,  and  this  fact  provides  a  framework  for  systematically  gathering  data  about  the  extent 
of  the  use  of  practices  that  can  improve  project  repeatability  and  consistency.  Project  managers  and 
practitioners  who  adopt  this  approach  are  encouraged  to  share  their  results  with  the  project  manage- 
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merit  and  CMM  communities  to  help  evolve  the  understanding  of  the  appropriateness  of  this  usage  of 
generic  practices. 
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TACOM-ARDEC  WHITE  PAPER 


SUBJECT:  The  Road  to  CMMI:  What  Works,  What’s  Needed? 

PURPOSE:  Describe  mechanisms  the  US  Army  TACOM-ARDEC  LCSEC  and 
STAR  organizations  developed  to  implement  CMMI  and  summarize  the  experiences 
resulting  from  transition  to  CMMI. 

ORGANIZATIONAL  ENVIRONMENT  FOR  PROCESS  IMPROVEMENT:  The 

process  improvement  activity  described  in  this  paper  involves  two  related  software 
support  organizations  located  at  TACOM-ARDEC,  Picatinny  Arsenal,  NJ.  These  two 
organizations  were  separately  implementing  process  improvement  programs.  To  im¬ 
plement  CMMI,  these  two  organizations  merged  their  strategic  improvement  objec¬ 
tives  and  conceived  a  unified  organizational  strategy  that  would  speed  their  process 
improvements  and  address  full  implementation  of  the  emerging  CMMI. 

TACOM  Life  Cycle  Software  Engineering  Center  (LCSEC).  Provides 
software  development,  software  configuration  management,  and  post  deployment 
maintenance  support  for  DOD  weapon  systems  and  training  devices.  Provides 
software  engineering  and  acquisition  support  to  Program  Managers  (PMs)  during 
system  acquisition. 

QED  System/Software  Technoiogy,  Analysis  and  Reliabiiity  (STAR) 
Team.  Performs  software  and  system  quality  assurance  functions.  Develops 
nondestructive  test  technology  and  measurement  technology.  Focuses  on  quality 
analysis  and  reliability  for  propellants,  explosives,  fire  control,  pyrotechnics,  tools 
and  equipment. 

The  combined  organizations  have  166  personnel  (91  Government,  75  contractor) 
that  perform  primarily  software  engineering  and  support  tasks.  The  product  areas 
supported  by  both  organizations  include  Artillery,  Munitions,  Trainers,  and  Combat 
Vehicles. 

USEFUL  MECHANISMS: 

1 .  Unified  Transition  Strategy.  The  unified  strategy  created  by  the 
LCSEC/STAR  organizations  was  focused  on  the  smooth  transition  from  existing 
processes  to  a  unified,  integrated  CMMI  implementation. 

Designed,  developed  and  implemented  a  set  of  common,  standard 
processes  (Common  Policies  and  Common  Procedures)  that  track  to 
the  evolution  from  the  SW-CMM,  SA-CMM,  and  SE-CMM  to  CMMI.  Re¬ 
vised  and  expanded  the  scope  and  methodology  of  the  existing  policies  to  ad¬ 
dress  the  requirements  of  CMMI.  Developed  traceability  matrices  (CMMs/CMMI  - 
Standard  Processes)  and  performed  a  Gap  Analysis  to  ensure  that  all  CMMI 
practices  were  addressed.  Ensured  that  the  set  of  policies  and  procedures  were 
jointly  implemented  in  both  organizations. 

included  available,  on-site  subject-matter  experts  in  the  planning  and 
implementation  efforts.  Integrated  CMMI  development  and  assessment 
knowledge  into  assessment  training  and  preparation  activities.  On-site  availability 
of  knowledgeable  experts  enabled  additional  insight  and  expert  input  into  the  de¬ 
velopment  and  acceptability  of  processes,  training,  coaching  and  planning. 
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Issues  addressed:  Inertia  and  cultural  resistance  to  changes  in  scope,  practices 
and  implementation  methodology  that  would  be  expected  from  changing  to  a  differ¬ 
ent  CMM  were  neutralized  by  this  evolutionary  transition  approach. 

Aids  to  CMMI  implementation:  Created  and  implemented  a  set  of  standard 
processes  (Common  Policies  and  Common  Procedures)  that  track  to  CMMI  prac¬ 
tices.  Based  all  training  and  coaching  efforts  on  this  set  of  policies  and  CMMI  prac¬ 
tices. 

Benefits  derived:  Positioned  the  organizations  to  participate  in  a  CMMI  pilot  as¬ 
sessment,  with  those  attendant  benefits.  Avoided  rework  and  non-productive  efforts 
expected  from  continuing  to  implement  separate  programs  and  processes. 

Dissemination:  Posted  the  set  of  standard  processes  and  supporting  documents 
on  the  Process  Asset  Library  (PAL).  Defined  the  strategy  in  a  unified  Capstone 
document.  Briefed  the  strategy  to  upper-level  management  and  all  organizational 
personnel. 

2.  Organizational  Issues:  The  unified  strategy  facilitated  the  common  application 
of  resources  for  joint  projects  and  accelerated  the  institutionalization  of  the  standard 
processes  by  both  organizations.  This  unified  application  also  enabled  the  adoption 
and  adaptation  of  CMMI  to  non-developmental  projects. 

Adopted  and  implemented  the  Software  Enterprise  concept,  based  on 
a  set  of  common  processes,  modeled  on  CMMI.  Capitalized  on  the  two 
organization’s  complimentary  missions  and  functions.  Addressed  supporting  par¬ 
ticipation  in  joint  projects.  Designed  a  common  senior  management  review  activ¬ 
ity.  Combined  the  Process  Engineering  Groups  (PEG)  from  each  organization  into 
a  single,  unified  PEG.  Improved  our  ability  to  work  together;  synergized  and  ac¬ 
celerated  the  improvement  process. 

Integrated  the  Quality  Assurance  (Process  Assurance  and  Product 
Evaluation)  function  across  both  organizations.  Applied  CMMI  (vice  the 
SW-CMM)  to  foster  a  better,  more  unified  definition  and  implementation  of  the 
quality  assurance  functions;  improved  the  communication  and  cooperation  be¬ 
tween  the  organizations;  and  eased  the  integration  of  the  unified  partnership. 

Applied  CMMI  to  non-development  projects  (e.g.,  service,  iV&V,  test). 

Using  CMMI  permitted  the  application  of  the  standard  processes  to  all  LCSEC 
and  STAR  projects,  which  increased  and  broadened  the  scope  of  management 
insight.  Including  all  projects  provided  added  insights  into  the  tailored  application 
of  CMMI. 

Issues  addressed:  Definition  of  a  mutual  and  supporting  approach.  Adjusting  the 
two  improvement  programs  to  support  common  objectives.  Commonality  and  unity  of 
purpose.  Coverage  of  service  related  projects. 

Aids  to  CMMI  implementation:  Unified  Capstone  document  that  established  and 
communicated  the  organizational  objectives.  Lessons  learned  and  experience 
gained  from  a  previous  pilot  assessment  of  CMMI  by  STAR,  and  a  dry  run  of  CMMI 
by  LCSEC. 

Benefits  derived:  Better  use  of  project  resources.  Application  to  all  projects.  Mu¬ 
tual  standards  and  evaluations. 
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Dissemination:  Publication  of  the  Capstone  document.  Briefings  by  Senior  Man¬ 
agement,  the  unified  PEG  and  the  unified  QA  Project  to  all  personnel. 

3.  Management  Commitment  and  Communications:  Combined  resources 
and  strategic  commitments  from  both  organizations  focused  upper-level  manage¬ 
ment  attention  on  the  CMMI  program.  Capitalized  on  the  achievements  of  similar 
Army  organizations  to  foster  motivation  and  commitment  from  senior  and  upper-level 
management. 

Exploited  previous  difficulties  as  individual  organizations.  Used  defi¬ 
ciencies  uncovered  by  the  previous  assessment  and  dry  run  to  provide  the  pre¬ 
cept  and  impetus  for  renewed,  accelerated  and  unified  improvement  efforts. 

Used  successes  experienced  by  sister  organizations.  When  other  Army 
software  support  organizations  publicized  their  SW-CMM  successes,  this  publicity 
energized  senior  management  to  expand  their  vision  for  the  TACOM-ARDEC 
software  community;  enabled  the  upper-level  management  sponsorship  and  sup¬ 
port  that  was  critical  for  accelerated  implementation  of  CMMI. 

Obtained  commitment  of  management  to  active  and  frequent  partici¬ 
pation  and  involvement.  The  senior  managers  initiated  and  implemented 
weekly  senior  management  meetings  to  specifically  address  the  status  and  pro¬ 
gress  of  the  unified  process  improvement  program.  The  TACOM-ARDEC  upper- 
level  management  also  instituted  a  set  of  special  in  process  reviews  (IPRs)  to  ad¬ 
dress  progress  toward  achievement  of  CMMI  Levels  2-4.  These  added  initiatives 
served  to  emphasize  to  all  personnel  the  importance  placed  by  management  on 
the  unified  program  and  improve  the  response  and  support  from  the  LCSEC  and 
STAR  personnel. 

Developed  a  unified  PEG  workshop  methodology,  expanded  mem¬ 
bership  activity  and  impiemented  a  unified  improvement  schedule. 

Established  a  regular  twice-monthly  PEG  meeting  schedule.  Designed  the  mem¬ 
bership  to  include  the  active  participation  of  the  senior  managers  and  project 
leaders  as  PEG  members.  Used  this  forum  to  both  communicate  require¬ 
ments/status,  and  to  obtain  periodic  feedback,  consensus  and  buy-in. 

Issues  addressed:  Sponsorship  and  support  from  management.  Communication 
of  unity  of  purpose  and  commitment  to  all  organizational  personnel.  Active  involve¬ 
ment  and  buy-in  from  all  levels  of  management. 

Aids  to  CMMI  implementation:  PEG  schedule  and  corrective  action  plan, 
mapped  to  CMMI  practices,  that  addressed  the  results  of  the  previous  pilot  and  dry 
run.  Status  briefings  and  PEG  reports  that  addressed  the  status  of  CMMI  implemen¬ 
tation. 

Benefits  derived:  Improved  communication,  feedback  and  buy-in  by  all  levels  of 
the  organization. 

Dissemination:  The  PEG  Workshop  Proceedings  posted  and  e-mailed  to  all  mem¬ 
bers.  Results  of  upper-level  management  briefings  provided  to  all  personnel. 

4.  Artifacts  and  Toois:  Modified  existing  tools  and  artifacts  from  each  organiza¬ 
tion  to  address  the  practices  of  CMMI,  then  transitioned  and  made  them  available  for 
both  organizations.  Applied  the  resources  of  both  organizations  to  the  identification 
and  development  of  added  framework  elements  and  artifacts  to  address  CMMI. 
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Established  combined  Process  Asset  Libraries  (PAL)  and  data 
repositories.  Developed  a  set  of  web  pages  and  repositories  for  storage  and 
access  to  process  improvement  information.  Made  this  information  readily 
available  to  all  LCSEC  and  STAR  personnel.  Expanded  the  use  to  both 
organizations  of  the  existing  CM  Repository  for  storage  and  control  of  both  project 
and  baseline  information.  Also  expanded  and  adopted  an  existing  DM  Repository 
for  electronic  storage  and  ready  access  of  project  work  products. 

Developed  additional  framework  elements  and  implementation  aids. 

Developed  a  set  of  procedures  (organizational  and  project),  standards,  templates, 
checklists,  forms,  briefing  slides,  and  guides  for  use  by  the  project  personnel. 
These  added  framework  elements  provided  an  added  commonality  and  stan¬ 
dardization  to  the  work  product  development  and  management  activities. 

Identified  and  collected  standards  and  examples  of  work  products 
and  artifacts.  Expanded  on  the  information  contained  in  CMMI  practices  to  pro¬ 
vide  added  guidance  for  development  and  maintenance  of  artifacts.  Identified  ex¬ 
amples  of  acceptable  artifacts  and  incorporated  them  into  the  PAL.  Identified  spe¬ 
cialized  work  products  required  by  unique  ARDEC  support  infrastructure 
(Training,  Procurement,  Materiel  Release)  and  incorporated  examples  into  the 
PAL. 

Integrated  the  Quality  Assurance  audit  process  with  CMMi  require¬ 
ments  and  practices.  Established  a  unified  QA  Project  and  a  set  of  checklists 
that  audited  compliance  with  the  set  of  organizational  processes  and  provided 
traceability  to  CMMI.  Provided  independent  input  as  to  the  state  of  project  imple¬ 
mentation  of  CMMI;  provided  early  warning  of  areas  needing  improvement,  and 
monitored  the  corrective  action  plans. 

Issues  addressed:  Common  documentation  structures.  Acceptable  quality  of  arti¬ 
facts  and  project  records.  Standard  application  and  use  of  tools. 

Aids  to  CMMI  implementation;  Common  PAL,  which  provided  organized  access 
to  process  elements  (both  standard  and  tailored).  The  set  of  audit  checklists  that 
provided  traceability  to  CMMI. 

Benefits  derived:  Common  training  and  coaching  base.  Uniform  assessment  of 
standards  compliance  and  work  product  usage. 

Dissemination:  PAL,  CM  Repository  and  DM  Repository.  QA  Project  Audit  Re¬ 
ports. 

5.  Training  Initiatives:  Used  personnel  trained  in  CMMI  and  on-site  subject  mat¬ 
ter  experts  to  develop  and  implement  CMMI  training  tailored  to  the  organization.  By 
using  these  resources,  completed  a  series  of  training  and  coaching  sessions  in 
much  less  time  and  with  minimal  coordination. 

Created  a  set  of  questions  and  answers  that  address  CMMI  practices. 

Developed  different  shreds  of  CMMI  that  track  to  the  various  organizational  and 
project  roles  in  the  LCSEC  and  STAR  organizations.  Used  these  to  help  the  per¬ 
sonnel  understand  the  requirements  for  and  meaning  of  CMMI. 

Accomplished  a  pilot  assessment  of  the  draft  CMMI  (STAR)  and  an  in¬ 
ternal  dry  run  (LCSEC).  Built  on  joint  commitment  to  implement  CMMI,  and 
the  assessment  and  dry  run,  to  secure  senior  management  sponsorship  to  par- 
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ticipate  in  a  CMMI  V1 .02d  pilot  assessment.  Exploited  SEI  participation.  Imple¬ 
mented  corrective  actions  identified  by  the  previous  pilot  and  dry  run,  to  acceler¬ 
ate  our  process  improvement,  as  measured  against  CMMI. 

Designed  a  specially  targeted  training  and  coaching  program.  De¬ 
signed  and  implemented  a  set  of  training  sessions,  training  aids,  and  coaching 
sessions  that  were  specifically  targeted  to  Improving  the  knowledge  and  under¬ 
standing  of  CMMI  requirements  and  practices,  as  implemented  in  the  set  of  poli¬ 
cies  and  procedures.  Trained  and  used  the  set  of  policies  and  procedures  as  the 
basis  for  the  process  improvement  efforts. 

Acquired  additional  CMMI  Training.  Arranged  for  CMMI  training  courses  tar¬ 
geted  to  the  organization.  This  training,  arranged  for  and  supported  by  our  on-site 
subject  matter  experts,  enabled  an  accelerated  understanding  and  application  of 
CMMI  requirements  and  practices. 

Issues  addressed:  Familiarity  and  understanding  of  CMMI  practices  by  all  per¬ 
sonnel.  Identification  of  areas  needing  extra  management  attention  and  emphasis. 

Aids  to  CMMI  implementation:  On-site  expertise  and  training.  Training  slides 
and  courses  that  addressed  CMMI  and  the  unified  process  framework. 

Benefits  derived:  Accelerated  level  of  understanding  and  knowledgeable  applica¬ 
tion  of  CMMI.  Able  to  focus  resources  on  improving  weak  areas. 

Dissemination:  On-site  training  courses.  Posted  training  information  and  material 
in  the  PAL. 

SUMMARY: 

Lessons  Learned: 

1 .  Active  senior  managerhent  involvement  is  prerequisite.  Periodic  meetings  must 
be  established  to  maintain  this  active  participation. 

2.  Communication,  training  and  coaching  are  all  essential  and  must  be  inte¬ 
grated. 

3.  CMMI  and  the  standard  processes  must  be  supplemented  by  an  extension  of 
the  framework  to  include  templates,  forms,  and  checklists  for  use  by  the  projects. 

4.  The  membership  of  the  PEG  needs  to  be  expanded  to  include  all  of  the  project 
leaders. 

5.  CMMI  does  not  address  the  quality  of  the  framework,  all  of  the  activities  or  all 
of  the  work  products.  The  organization,  and  the  PEG  especially,  needs  to  be  con¬ 
stantly  sensitive  to  the  quality  of  the  implementation. 

6.  CMMI  needs  to  be  tailored  for  application  to  non-developmental  projects. 

7.  A  central  PAL  and  repository  are  basic  and  must  be  actively  stocked  and  main¬ 
tained. 

Successful  Practices: 

1 .  Active  integration  of  the- quality  assurance  function  into  the  process  improve¬ 
ment  effort,  especially  targeting  the  audits  to  assess  compliance  with  CMMI  as  re¬ 
flected  in  the  standard  processes. 
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2.  Extension  of  the  standard  processes  through  a  comprehensive  set  of  frame¬ 
work  elements,  such  as  templates,  forms,  checklists  and  training  information. 

Transition-related  Benefits: 

1 .  CMMI  is  less  burdensome  in  the  implementation  phases.  For  example,  the 
extensive  set  of  specific  documented  procedures  required  by  SW-CMM  have 
been  eliminated,  permitting  a  more  tailored,  economic  development  of  the  stan¬ 
dard  processes  and  procedures. 

2.  CMMI  provides  detailed  integrated  acquisition  management  coverage  that  was 
only  partially  available  in  the  SW-CMM. 

3.  Ability  to  apply  CMMI  to  non-developmental  projects  increases  “bang  for  the 
buck”. 

Authors: 

Wayne  Sherer,  TACOM-ARDEC  FC  &  SED,  Senior  Technical  Associate  for  Cor¬ 
porate  Process  Improvement 

Mary  Gregg,  TACOM-ARDEC  LCSEC,  PEG  Leader 

Chuck  Gordon,  Anchor  Software  Management,  PEG  Facilitator 

Alison  Ferraro,  TACOM-ARDEC  QED  STAR,  PEG  Leader 
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The  facilitator  was  Mike  Konrad.  The  scribe  was  Lynn  Penn.  The  recorder  was  Diane  Gib¬ 
son.  Julie  Barnard  and  Bruce  Boyd  volunteered  to  be  the  report  editors.  Julie  Barnard  pre¬ 
sented  the  working  group’s  findings  to  the  workshop.  Note  that  there  were  two  separate 
working  groups  on  the  CMMI®*^  topic  at  the  workshop,  due  to  the  level  of  interest  in  the  topic 
expressed  by  the  workshop  attendees.  This  CMMI^*^  working  group  (1.5)  was  the  first  of  the 
two  working  groups  convened  on  the  topic.  Different  participants  were  involved  in  each  of 
the  two  CMMI®^  working  groups. 

Hypotheses  and  Observations 

This  section  discusses  the  observations,  hypotheses,  and  propositions  that  initiated  the  dis¬ 
cussion.  Also  included  are  the  results  of  brainstorming  activities  that  did  not  become  a  work¬ 
ing  group  consensus. 

The  working  group  brainstormed  the  following  set  of  initial  questions  to  be  discussed: 

■  How  does  an  existing  high  maturity  software  organization  integrate  with  a  relatively 
immature  systems  engineering  organization  when  transitioning  from  Software  CMM® 
to  CMMI? 

■  What  is  the  next  step  for  CMMI^”^  development  and  release?  How  do  we  plan  for 
CMMI®'**  over  the  next  two  years? 
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■  How  will  help  organizations  that  develop  custom  software?  How  will  CMMl, 

which  covers  systems  engineering,  help  those  who  don’t  see  systems  engineering  as  part 
of  their  business?  How  practical  are  assessments  if  they  can  take  2-3  weeks? 

■  Where  can  we  find  mapping  from  Software  CMM®  to  for  higher  maturity 

level  organizations? 

■  What  are  the  qualifications  for  SCAMPI  assessors?  How  long  do  assessments  take? 

■  How  do  you  integrate  a  number  of  separate  legacy  organizations  using  follow¬ 

ing  mergers?  How  difficult  is  it  to  cover  a  diverse  organization  with  a  common 

assessment? 

■  The  product  development  team  is  interested  in  hearing  the  concerns  of  indus¬ 
try  on  the  model.  How  do  you  apply  to  commercial  products  not  currently 

covered  by  CMMI? 

■  When  merging  various  companies  into  one,  shouldn’t  we  deal  with  the  merger  issues 
first,  then  CMMI?  Software  is  just  a  part  of  what  the  company  does  -  CMMl^”^  will 
cause  companies  to  pull  in  more  of  the  organization  than  just  engineering  into  their  im¬ 
provement  plans  -  for  example,  things  in  the  factory,  or  in  the  quality  side  of  the  house. 
We’re  implementing  Level  4  in  software  for  commercial  production  because  we  know  it 
is  the  only  way  to  meet  the  contract.  We  would  like  to  do  similar  improvement  on  the 
engineering  side  of  the  house. 

■  We  are  currently  Level  5  with  both  Software  CMM®  and  Systems  Engineering  CMM®. 
We  are  currently  in  the  rollout  and  integration  of  CMMI^”^.  We  were  also  member  of  the 
CMMI^'’’  working  group  previously.  We  want  to  see  how  CMMI^^  has  evolved  and  get 
results  from  the  pilot  assessments.  We’d  also  like  to  hear  interpretations  of  the  continu¬ 
ous  vs.  staged  representations. 

■  SEI  would  like  to  understand  what  high  maturity  organizations  think  about  the  practices 
at  Levels  4  and  5  in  CMMI^”^  version  1.01.  How  different  are  they  from  Version  1.1  of 
Software  CMM®?  How  much  is  enough  to  be  assessed  at  Levels  4  and  5?  How  much 
of  the  product  life  cycle  needs  to  be  brought  under  statistical  process  control? 

■  We  have  a  similar  question  regarding  Technology  Change  Management  —  When  high 
maturity  organizations  evaluate  and  decide  to  adopt  new  technology,  is  that  activity  sup¬ 
posed  to  be  under  Statistical  Process  Control  (SPC)?  Since  technology  is  changing 
quickly  and  changes  are  happening  so  fast,  should  it  be  [under  SPC]? 

■  What  are  the  lessons  learned  that  could  be  used  by  a  novice  organization  applying 
CMMI^*^  vs.  an  organization  experienced  with  Software  CMM®? 

■  Where  is  CMMl^"^  going  in  the  future?  Will  it  include  the  People  CMM?  Will  it  apply 
to  an  Information  Technology  organization?  Will  there  be  one  assessment  for  entire  or¬ 
ganization? 

■  What  experiences  have  people  had  with  CMMI^"^  lessons  learned?  What  about  extend¬ 
ing  the  CMM®  to  other  areas? 

■  How  do  you  perform  Statistical  Process  Control  for  areas  other  than  software  specific 
development? 

Several  working  group  members  asked  for  background  information  on  CMMI^*^.  Mike  Kon¬ 
rad  (SEI)  provided  a  brief  summary  of  relevant  CMMI^'^  information  to  the  group.  Since  this 
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information  is  readily  available  from  the  SEI  website  and  other  sources,  it  is  not  reproduced 
in  this  report.  Some  of  the  members  asked  about  mapping  of  various  other  models  and  stan¬ 
dards  to  It  was  noted  that  some  useful  mappings  are  available  on  the  Software 

Technology  Support  Center  (STSC)  website  http://www.stsc.hill.af.mil/.  Additionally,  the 
SEI  web  site  includes  pointers  to  the  same  mapping  documents  resident  on  the  STSC  web 
site  that  compare  the  Software  CMM®  to  the  CMMI^'^  and  vice-versa. 

During  initiation  of  the  working  group,  the  working  group  members  posed  the  following 
questions  or  observations: 

Observation  #1:  Are  software  organizations  using  CMMI? 

It  was  argued  that  software  engineering  and  system  engineering  are  not  really  separate  disci¬ 
plines  -  either  or  both  have  been  characterized  as  encompassing  the  other.  Engineering 
process  areas  talk  about  defining  the  processes  that  go  with  developing  and  using  the  product: 
manufacturing,  customization,  training,  repair,  etc.  A  broader  view  of  life  cycle  is  required  - 
for  example,  maintainers  of  products  who  know  all  the  effort  required  to  make  a  product  use¬ 
ful  and  keep  it  functional.  CMMI^*^  gives  more  attention  to  these  stages.  Organizations  that 
are  only  developing  software  still  have  integration  issues  about  installation,  help  desk,  tech 
support,  etc.  CMMI^*^  gives  software  organizations  that  develop  applications  a  model  to  in¬ 
clude  other  aspects  of  product  development;  e.g.,  relevant  stakeholders.  CMMI^''^  practices 
integrate  more  decision-making  and  parts  of  product  development.  The  value  of  SW-CMM® 
was  to  give  focus  to  neglected  areas  such  as  support  and  project  management.  CMMI^"^ 
folds  in  lessons  learned  from  SW-CMM®  and  from  engineering  -  EIA  731.  CMMI^*^  tried 
to  capture  these  lessons  learned.  At  Software  CMM®  Maturity  Levels  4  and  5  one  of  the 
lessons  learned  was  that  the  Technology  Change  Management  and  Process  Change  Manage¬ 
ment  Key  Process  Areas  could  be  merged  (from  the  workshop  on  TCM);  also,  product  lines 
were  included  in  SW-CMM®  Version  2.  There  is  still  an  opportunity  to  look  upstream, 
downstream  and  laterally  for  information  about  the  products  and  services  an  organization 
provides. 

The  issue  that  the  working  group  discussed  in  more  detail  was: 

Marketing 

■  Commercial  organizations  have  different  issues  from  defense  contractors;  for  example, 
marketing  is  so  much  more  important.  Does  CMMI®*^  focus  more  on  marketing,  or 
might  it? 

■  System  engineering  issues  -  customer  requirements,  product  management,  etc.  -  often 
focus  on  information  needed  by  marketing.  Someone  is  working  on  a  Masters  thesis  on 
a  CMM®  for  marketing. 

■  Does  the  model  sub-optimize  the  commercial,  marketing  approach? 

■  CMU  is  working  with  private,  commercial  companies  -  using  the  CMM®  as  a  focal 
point  (e.g..  Sun,  Adobe,  3Com,  Oracle). 

■  Look  at  the  participants  in  the  development  of  CMMI®*^  -  most  were  defense  contrac¬ 
tors.  -  but  there  were  some  commercial  companies  -  Motorola,  Ericsson  -  all  were  ei¬ 
ther  defense  or  telecommunications  companies.  SW-CMM®  began  as  a  tool  for  defense 
contractors  with  SEI  shepherding  the  flock.  SEI  wants  input  from  commercial  organiza¬ 
tions,  but  SW-CMM®  is  something  for  software  development  -  marketing  folks  are  not 
very  excited  about  it.  It  would  be  better  if  there  were  something  that  comes  from  mar¬ 
keting  to  get  them  involved.  CMMI®*^  has  stakeholder  involvement;  i.e.,  coordination 
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with  the  stakeholders  group;  but  it  suffers  from  not  having  an  explicit  “marketing  the 
product”  process  area.  Maybe  this  will  happen  in  the  future? 

■  Business  acquisition  is  a  major  focus  of  companies.  Requirements  are  important,  but 
also  need  the  business  acquisition  process  (business  development,  marketing)  to  focus 
on  risks,  etc.  CMMs  give  a  push  to  engineering  but  not  to  marketing. 

•  When  we  bring  software  or  engineering  improvement  to  the  boardroom,  CMM®  seems 
parochial.  They  are  more  interested  in  growing  the  business  and  making  profits.  We 
need  to  bring  software  improvement  back  into  improving  the  business  and  addressing 
business  issues. 

■  People  are  making  the  argument  that  CMMI^*^  has  to  be  translated  for  specific  business 

contexts.  If  constrains  ability  to  meet  business  goals,  we  would  like  to  hear 

more  examples  or  evidence  of  this. 

■  has  weaknesses  in  the  areas  of  marketing  and  business  development.  What¬ 
ever  the  SEI  wants  to  address  regarding  marketing  needs  to  be  clearly  addressed  in  train¬ 
ing  for  instructors  and  for  assessors.  We  don’t  expect  this  to  really  change  in  next  3 
years. 

■  Is  there  a  set  of  principles  that  can  be  used  as  guidelines  in  other  parts  of  organization 
that  are  outside  the  CMMI?  For  example,  are  there  architectural  principles  for  adding 
new  disciplines,  or  new  generic  practices?  Is  there  a  single  model  for  product  develop¬ 
ment  processes  /  organizations  and  a  path  for  adding  other  disciplines  and  application 
environments? 

■  We  are  looking  to  members  of  this  group  for  people  who  are  applying  CMMI^*^  in  the 
commercial  world  and  in  other  areas  for  their  insights. 

Observation/  Question  #2:  Standard  Assessment  Method  for  Process  Im¬ 

provement  (SCAMPI)  issues 

■  Initial  concern  -  SCAMPI  takes  such  a  long  time  and  is  an  intense  effort.  Today  it  takes 
about  100  hours  (clock  time)  over  9  days.  The  second  time  a  lead  assessor  takes  less 
time.  It’s  getting  more  like  CMM®  Based  Appraisal  for  Internal  Process  Improvement 
(CBA-IPI),  plus  there  are  some  innovations  that  could  also  be  used  in  CBA-BPI. 

■  Concern  with  CMMI^'^  -  It  is  possible  (likely?)  that  Dr.  Etter  will  change  the  require¬ 
ment  for  SW-CMM®  Level  3  to  CMMI®”^  Level  3  or  something  equivalent.  How  as¬ 
sessments  are  done  is  critical  here. 

Observation/  Question  #3:  CMMI®"^^  changes 

■  CMMI^*^  Version  1.1  is  expected  to  be  released  in  December,  and  then  be  stabilized  for 
four  or  more  years.  The  desire  is  that  V  1.1  will  be  similar  enough  to  V  1.0  that  folks 
won’t  need  to  be  retrained.  No  changes  are  expected  in  a  number  of  process  areas;  goals 
and  practices  should  be  pretty  much  the  same. 

■  Biggest  changes  will  be  in  the  evaluation  techniques  (SCAMPI).  From  Dr.  Etter’s  office 
-  there  will  be  a  “source  capability  evaluation”  method.  We  do  anticipate  changes  in  the 
assessment  method  to  save  time  and  take  advantage  of  lessons  learned  in  pilot  assess¬ 
ments. 

■  Regarding  which  new  disciplines  will  be  added  to  the  model:  there  is  no  clear  direction. 
Will  add  acquisition  in  some  form;  security  is  pushing  also;  enterprise  modeling,  pro- 


70 


CMU/SEI-2001-TR-015 


gram  office,  etc.  are  all  being  raised  but  no  decision  has  been  made.  People  CMM®  Ver¬ 
sion  2  is  being  crafted  to  be  compatible  with  CMMI^'^  -  there  have  been  joint  assess¬ 
ment  of  People  CMM®  and  CMMI^^. 


Observation/  Question  #4:  Expansion  to  other  areas 

■  One  company  tried  to  include  other  disciplines  using  the  SW-CMM®  -  but  when  senior 
management  heard  about  CMMI^”^  they  stopped  that  initiative.  Moving  improvement 
into  product  development  processes,  which  is  more  than  just  integrated  engineering 
processes. 

Observation/  Question  #5:  SPC 

■  CMMI^*^  promotes  doing  SPC  where  the  business  case  suggests  you  need  SPC.  If  TCM 
is  very  important  to  an  organization,  then  they  might  want  to  use  SPC  for  TCM  -  in 
other  organizations,  this  may  not  be  needed. 

■  How  much  SPC  is  enough?  Are  there  any  universal  processes  that  should  always  be 
placed  under  SPC? 

Issue  A:  How  will  CMMI^’''*  apply  to  a  novice  organization  vs. 
one  with  a  mature  software  organization?  If  you  have 
several  pockets  of  high  maturity  practices,  how  do 
you  apply  CMMI^*^  to  additional  areas? 

Examples  of  high-maturity  organization  experiences; 

■  System  engineers  wanted  to  learn  and  adopt  processes  used  by  software  folks  when  they 
participated  in  Integrated  Product  Teams  (IPTs)  with  Level  5  software  engineers. 

■  An  executive  commented  that  their  company  no  longer  received  complaints  about  soft¬ 
ware,  but  they  still  got  complaints  about  other  engineering  domains  -  so  they  tried  to 
apply  CMM®  principles  to  top  level  product  development  processes,  including  engi¬ 
neering,  business,  and  end-to-end  processes.  They  intend  to  apply  CMMI®*^  at  some 
point.  This  was  an  example  of  executive  push.  Having  demonstrated  the  benefits  of 
CMM®  in  software,  they  wanted  to  apply  it  more  broadly.  It  was  somewhat  awkward  to 
apply  CMM®  outside  of  software,  but  it  was  a  valuable  exercise.  There  was  previously 
no  concept  of  peer  reviews  of  business  plans,  but  the  practice  was  introduced  because  it 
made  sense  —  they  were  important  documents. 

■  Experience  getting  ready  for  a  CMMI^"^  pilot:  It  was  recognized  that  SW-CMM®  had 
added  value  and  been  beneficial  to  the  software  community,  so  engineering  management 
was  ready  to  try  it,  even  though  they  didn’t  know  exactly  what  it  was.  They  saw  real 
benefits  from  the  Level  3  aspects,  rather  than  the  high  maturity  aspects. 

■  Another  organization  doesn’t  have  mature  system  engineering  and  has  software  engi¬ 
neering  communities  that  are  diverse.  There  are  up-front  components  and  back-end 
components  that  haven’t  paid  enough  attention  to  process  improvement.  When  SE- 
CMM®  came  out,  they  looked  for  areas  to  piggyback  usage  between  software  and  sys¬ 
tem  engineering  -  where  they  could  use  software  processes  in  the  system  engineering 
world.  System  engineering  processes  became  part  of  the  improvement  structure.  Total 
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Quality  Management  (TQM)  also  provided  some  foundational  principles  that  extended 
across  disciplines  -  TQM  was  highly  leveragable.  Qual  tech,  TQM  approach  was  ap¬ 
plied  across  the  organization  first,  then  they  applied  SW-CMM®. 

■  Another  organization  matured  in  both  system  engineering  and  SW-CMM®.  They  re¬ 
placed  SEPGs  with  an  Engineering  Process  Group,  with  members  from  software,  system 
engineering,  quality,  configuration  management,  business  development,  process  im¬ 
provement,  etc.  They  needed  this  joint  structure  to  achieve  joint  improvement.  The  re¬ 
sulting  processes  are  credible  because  they  have  a  representative  from  each  area  in  the 
group.  When  they  establish  processes  in  software,  they  need  involvement  of  estimators, 
managers,  etc.  It  is  beneficial  to  put  all  groups  together  in  the  process  group  so  they  de¬ 
fine  processes  together,  and  allow  for  a  more  natural  progression.  This  was  the  biggest 
benefit:  to  have  everyone  work  together.  As  they  pulled  new  organizations  into  the  com¬ 
pany  (by  merger),  the  process  group  was  able  to  deal  with  the  mapping  and  the 
processes  in  the  new  organization. 

Question:  If  an  organization  is  starting  from  scratch  (no  previous  CMM®  experience),  do 
you  recommend  first  focusing  on  software  and  then  including  other  engineering  areas?  Or 
should  you  go  for  them  all  concurrently? 

■  One  organization  embarked  on  an  enterprise  level  improvement  effort  by  first  getting 
software  to  Level  3.  They  wished  they  had  done  it  all  together  from  the  start.  Software 
had  a  lot  invested  in  their  approach,  and  had  to  convince  others  that  their  way  was  good 
for  all. 

Conclusion:  Depending  upon  the  organizational  circumstances  there  were  examples  de¬ 
scribed  that  support  both  positions  --  software  can  be  an  inspiration  for  other  parts  of  the  or¬ 
ganization,  or  it  can  be  best  to  introduce  change  across  the  entire  engineering  organization  at 
one  time. 

Another  organization  implemented  ISO  9001  first,  then  SW-CMM®,  then  TQM  in  1995. 
They  included  Human  Resources  and  other  areas,  including  marketing,  into  their  TQM  im¬ 
plementation.  They  will  limit  the  use  of  CMMI^*^  while  still  looking  at  the  enterprise 
through  TQM  and  benchmarking. 

CMMI^”^  reinforces  the  shared  model  -  defining  high  leverage  process  areas  for  all  of  the 
organization.  It  highlights  commonality  and  areas  for  integration. 

If  an  organization  is  looking  at  leveraging  high  maturity  experience  to  areas  with  no  maturity, 
the  CMMI^'^  Generic  Practices  (GPs)  are  the  basic  behavioral  principles  that  can  be  used  in 
any  discipline.  They  become  a  model  to  use  in  every  discipline. 

The  set  of  GPs  is  a  candidate  for  high  leverage  processes  for  different  areas.  Generic  prac¬ 
tices  can  be  used  in  many  areas  for  process  improvement. 

Issue  B:  Applying  CMMI^^  to  additional  areas?  What  is  the 
effect  of  organizational  mergers  on  high  process 
maturity? 

Some  examples  follow  of  high-maturity  organization  experiences  with  acquisitions,  mergers, 
and  reorganizations  and  its  associated  effect  on  blending  processes  and  process  maturity: 
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■  One  large  company  of  approximately  8500  people  integrated  another  organization  of  ap¬ 
proximately  2000  people  together  through  a  merger.  In  discussions  on  the  process  stan¬ 
dard  to  be  used  for  the  newly  formed  organization  of  10,000  people,  the  company  level 
process  group  began  to  talk  about  what  level  of  process  commonality  should  exist  across 
the  organization.  The  standard  process  existed  prior  to  the  merger;  however,  it  was  rec¬ 
ognized  that  the  existing  process  standard  might  not  be  immediately  achievable  by  the 
new  parts  of  the  organization.  Representatives  from  the  new  part  of  the  organization 
participated  in  the  company  process  group  to  review  the  process  standard.  The  com¬ 
pany  process  standard,  which  is  the  set  of  minimum  standard  processes  to  be  used  for  all 
organization,  covers  20  processes.  Each  process  is  represented  in  about  1  page  of  struc¬ 
tured  English  text  and  is  task  oriented.  As  a  result  of  the  merger,  the  process  standard 
was  modified  so  that  the  level  of  detail  was  raised  for  the  newly  formed  organization  to 
something  that  everyone  could  live  with,  that  their  supporting  procedures  could  support, 
and  that  everyone  across  the  organization  could  use  the  process  standard.  As  the  revised 
higher-level  standard  was  adopted  and  deployed  by  the  newer  group,  then  the  company 
process  group  could  revisit  the  level  of  detail  in  the  standard.  The  process  standard  be¬ 
gan  to  get  more  detailed  as  more  parts  of  the  organization  used  similar  processes.  At  a 
later  point  in  time  when  a  second  new  group  was  incorporated  into  the  company,  the  re¬ 
view  of  the  processes  began  again  to  determine  what  lowest  common  denominator  of 
standard  process  could  be  accepted  across  the  board.  The  detailed  process  information 
was  captured  and  retained  during  these  revision  periods  so  that  the  processes  could  be 
tailored  subsequently  and  include  the  details  as  appropriate. 

•  In  another  organization  of  about  2200  software  people,  they  tried  a  similar  approach 
when  affected  by  a  merger.  One  of  their  big  struggles  was  with  the  customer  Defense 
Contract  Management  Agents,  who  report  to  different  program  offices.  The  program  of¬ 
fices  were  resisting  the  standardization  of  processes,  because  they  are  site  focused.  They 
are  comfortable  with  the  way  things  are  and  don’t  want  to  see  changes.  In  this  case,  if 
the  customer  were  allowed  to  drive  the  standardization,  then  there  could  be  backward, 
instead  of  forward  progress. 

In  addition  to  the  software  process  impacts  associated  with  mergers,  there  was  discussion  of 
impacts  to  areas  such  as  Human  Resources,  marketing,  financial  practices  and  the  importance 
of  these  issues.  In  one  company,  there  was  focus  on  workforce  issues  (e.g.,  through  the  Peo¬ 
ple  CMM)  once  the  high  level  process  group  was  established  and  the  technical  processes  had 
achieved  a  high  level  of  maturity. 

In  one  organization,  the  affected  groups  had  to  evaluate  their  compliance  to  the  standard 
process  through  their  implementation  of  it.  This  included  use  of  tools  and  detailed  procedures 
in  implementation  of  the  standard  process.  For  example,  the  configuration  management  proc¬ 
ess  standard  contained  a  list  of  required  tasks.  Different  parts  of  the  organization  used  differ¬ 
ent  configuration  management  tools;  however,  as  long  as  the  tools  accomplished  the  required 
tasks  and  roles  in  the  high  level  processes,  then  there  was  no  need  to  change  tools.  If  the 
tools  in  use  did  not  accomplish  the  required  configuration  management  tasks,  then  that  part 
of  the  organization  needed  to  show  how  it  would  accomplish  all  of  the  required  tasks  from 
the  standard.  In  some  cases  this  resulted  in  a  change  of  tool  use.  New  projects  were  ex¬ 
pected  to  use  centrally  chosen  /  supported  tools.  A  similar  approach  was  used  in  evaluating 
the  compliance  of  low-level  procedures  and  their  support  of  the  high  level  process.  If  part  of 
the  organization  used  a  procedure  that  accomplished  all  the  required  standard  tasks,  then  it 
could  be  maintained.  Commonality  was  sought  where  it  made  sense.  For  example,  different 
kinds  of  peer  reviews  were  being  practiced  in  the  organization.  To  try  and  standardize  the 
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inspection  process,  a  formal  kaizen  event  was  conducted  on  inspections.  This  resulted  in  the 
same  form  of  inspections  being  adopted  throughout  the  organization. 

In  another  organization,  mapping  and  standards  were  in  place  across  the  new  organization, 
but  training  and  implementation  were  lagging  behind  a  bit.  In  addition,  there  was  some  resis¬ 
tance  to  things  like  quantitative  management.  So,  that  part  of  the  organization  does  not 
achieve  maturity  Level  4  in  the  targeted  time  period  (e.g.,  6  months.) 

One  organization  divests  itself  of  part  of  its  group.  The  Level  4/5  group  ended  up  trying  to 
maintain  their  maturity  level  through  ‘tribal  knowledge’.  They  winnowed  down  their  docu¬ 
mentation,  but  found  that  they  did  not  have  enough  detail  to  adequately  train  new  people. 

The  veterans  (“old  timers”)  loiew  the  process,  but  many  of  them  were  retiring.  This  caused  a 
need  to  re-document  their  processes.  They  struggled  through  a  Level  3  assessment  -  and  are 
climbing  back  up.  The  documentation  was  not  adequate  to  sustain  Level  4  when  they  lost  so 
many  people  who  had  institutionalized  processes  and  had  created  a  stable  organization.  This 
organization  recommended  that  documentation  be  evaluated  using  criteria  of  how  easily 
someone  can  pick  it  up  and  learn  to  do  the  processes. 

One  organization  represented  in  the  working  group  was  about  to  be  involved  in  a  merger  and 
was  seeking  suggestions  from  experienced  organization  for  what  it  takes  to  maintain  process 
maturity  during  take-over.  Some  ideas  provided  from  “merger-experienced”  organizations 
were  provided: 

■  Be  careful  about  how  senior  management  describes,  documents,  and  represents  the  take¬ 
over.  In  one  case,  the  combining  of  organizations  was  declared  to  be  a  merger  and  not  to 
be  a  take-over  and  that  the  best  was  to  be  combined  from  each  of  the  merged  organiza¬ 
tions.  However,  the  management  of  the  combined  organization  was  all  from  the  organi¬ 
zation  that  initiated  the  merger;  which  reflected  the  perception  of  take-over  rather  than 
merger.  Teams  need  to  be  established  as  soon  as  possible  and  should  begin  talking  be¬ 
fore  the  “thou  shall  do. . .”  is  issued  from  the  top  management. 

■  In  another  organization,  a  lot  of  time  was  spent  getting  to  know  other  process  people  in 
the  organization  to  establish  contacts  and  exchange  process  ideas.  This  resulted  in  not 
artificially  forcing  any  combinations. 

■  In  another  instance,  a  Level  5  organization  merged  into  a  Level  3  organization.  New 
people  began  to  work  with  and  help  the  Level  3  folks  with  their  issues.  They  worked 
toward  renaming  processes  -  no  one  retained  the  old  process  names,  but  rather  worked 
toward  creating  some  new  names  that  both  organizations  could  live  with  and  did  not 
convey  any  ties  to  the  past  organizational  structures. 

■  It  was  suggested  to  be  cautious  of  the  snob  factor  and  not  convey  one  organization  as 
“better”  than  the  other  even  when  there  are  differing  maturity  levels.  Ultimately,  there 
should  be  common  objectives  and  goals  that  unite  the  groups  and  the  “more  mature” 
groups  should  assist  the  “lesser  mature”  groups  and  do  so  in  humility. 

■  In  order  to  posture  a  high  maturity  company  so  that  the  impact  of  mergers  is  lessened,  it 
was  suggested  that  metrics  be  shared  with  new  organization  and  new  management  as  a 
part  of  the  familiarization  and  transition. 

Issue  C:  SOFTWARE  CMM®  and  CMMI®"^  -  key  differences  ? 

One  of  the  new  process  areas  in  CMMI^*^  is  Measurement  and  Analysis.  With  the  Software 
CMM®,  it  was  believed  that  there  wasn’t  enough  measurement  represented  at  lower  levels. 
Even  though  measures  existed  for  each  key  process  area,  they  were  often  considered  not  to  be 
useful  measures.  For  some  organizations  that  were  striving  toward  Level  4  maturity,  they 
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sometimes  had  to  rethink  a  lot  of  measures  that  were  put  in  place  at  lower  level  key  process 
areas.  Some  organizations  reported  that  they  did  not  wait  until  maturity  Levels  3  or  4  to  do 
process  measurement  and  quantitative  management,  but  rather  that  measurement  was  impor¬ 
tant  to  establish  at  lower  levels.  The  existence  of  Measurement  and  Analysis  is  one  of  the 
things  in  CMMI^'^  that  may  help  organizations  get  to  higher  maturity  levels  faster  by  provid¬ 
ing  the  necessary  foundation. 

Some  lower  maturity  level  organizations  may  not  see  the  need  for  metrics  because  they  are  so 
busy  just  trying  to  get  the  basics  done  and  can’t  consider  metrics.  Also,  projects  are  giving 
data,  but  may  not  see  benefits  until  the  Level  4  processes  are  established  and  they  receive 
information  back  on  how  to  do  things  better. 

One  organization  reported  that  they  performed  measurement  for  a  long  time.  However,  in 
1989  they  received  a  letter  demanding  improvement,  because  their  costs  were  too  high, 
schedules  were  unpredictable,  and  quality  was  poor.  They  performed  an  analysis  of  produc¬ 
ing  software  from  the  perspective  of  cost  and  schedule  -  that  was  better  than  focusing  on 
quality  (alone.)  They  established  a  solid  earned  value  management  process.  What  did  they 
see  in  the  Software  CMM®  at  that  time?  They  were  already  doing  measurement  -  so  they 
focused  on  quality  and  configuration  management  because  that  was  new.  is  saying 

that  measurement  is  important  at  lower  levels.  Organization  probably  shouldn’t  try  to  focus 
on  quality  alone,  but  rather  should  focus  on  cost  and  schedule  type  issues  as  well. 

Another  area  of  key  change  in  from  Software  CMM®  is  the  level  of  detail  of  the 

engineering  processes  for  product  development.  The  questions  were  raised: 

■  Is  that  detail  helpful  for  process  improvement  or  is  it  a  hindrance? 

■  Is  the  lack  of  focus  on  software  a  problem  or  a  benefit? 

Engineering  processes  can  be  used  to  demonstrate  what  might  be  needed  and  where  one 
might  begin  in  implementing  CMMI^*^.  The  CMMI^”^  represents  the  basics  of  engineering 
processes.  However,  there  may  be  problems  encountered  during  implementation  for  Informa¬ 
tion  Technology  organizations,  dot-coms,  and/or  shrink-wrap  organizations,  depending  on 
previous  experience,  types  of  software  development,  the  criticality  of  the  software  and  ex¬ 
perience  with  standards  in  the  past.  An  example  was  offered  of  a  backend  www  company 
with  no  existing  life  cycle,  no  sense  of  process  or  project  management  or  defect  tracking  yet 
CMMs  could  help  the  start-ups  if  used  well. 

The  CMMI*“  is  not  just  a  set  of  processes,  but  a  model  or  a  guide  to  improve  processes.  It 
appears  some  are  using  the  CMMI^”^  as  a  model  for  their  processes  rather  than  as  a  model  for 
measuring  ‘maturity’  of  processes.  Using  CMMI®*^  as  a  model  for  measuring  processes  -  is  a 
viewpoint  from  a  mature  organization.  So,  what  can  a  mature  organization  do  with 
CMMI^"^?  Do  they  have  to  rewrite  processes?  This  is  the  wrong  approach  -  it  is  not  a  set  of 
processes.  When  a  mature  organization  looks  at  a  new  model  and  tries  to  learn  from  it, 
somewhere  they  need  to  ask  if  what  they  have  is  adequate  or  is  there  something  missing. 
There  may  be  new  insights  or  ideas  coming  from  a  new  model  -  and  once  you  know  the  new 
idea,  you  want  to  implement  it  and  gain  advantage  from  it.  Improvement  can  come  from 
within  or  can  come  from  outside  the  organization. 

Transition  to  CMMI®"^  means  comparing  processes  against  a  new  model.  The  CMMI®'’^  em¬ 
bedded  some  of  what  was  learned  about  maturity  in  organizations  (e.g.,  PCM,  TCM,  and 
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OID).  Does  capture  the  paths  previously  taken  by  higher  maturity  organizations 

better  than  the  CMM®? 


Issue  D:  How  do  organizations  with  Software  CMM® 

experience,  but  no  CMM®  experience  in  systems 
engineering,  remove  stovepipes  to  impiement  CMMI? 

One  SW-CMM®  L4  organization  engaged  their  systems  engineering  people.  They  pulled 
together  their  processes  to  be  consistent  with  L3  software  processes  (on  one  particular  pro¬ 
gram).  Now,  some  system  engineering  groups  exhibit  LI  or  L2  behavior—  and  now  they  have 
to  deal  with  this.  There  are  clearly  defined  interfaces,  project-by-project,  program-by¬ 
program.  Software  processes  are  institutionalized,  but  interfaces  to  systems  engineering  are 
chaotic. 

Another  organization  has  two  levels  of  system  engineering  (aircraft  level  and  detailed  system 
level).  Those  areas  that  are  associated  with  software  have  adopted  some  of  the  software  prac¬ 
tices.  This  hasn’t  been  transferred  to  the  aircraft-level  system  engineers.  The  software  or¬ 
ganization  pulls  process  focused  behavior  -  and  the  spread  is  slow/resisted.  can 

bring  such  organizations  to  the  awareness  of  process-focused  needs  -  especially,  when  cus¬ 
tomers  say  they  are  going  to  use  to  evaluate  them. 

When  software  and  systems  are  tightly  coupled,  practices  do  diffuse  -  but  other  engineering 
areas  may  have  no  contact. 

In  one  organization,  the  software  process  owner  is  running  the  software  improvement  pro¬ 
gram  across  the  entire  organization.  The  engineering  process  improvement  effort  is  just  be¬ 
ginning  with  self-assessments,  documenting  processes,  and  evaluating  tools.  The  objective  is 
to  provide  measures  to  the  CEO  as  requested. 

Another  organization  described  a  leap  frogging  approach.  Software  engineering  was 
way  ahead  but  systems  engineering  was  working  with  the  software  processes.  When  the 
company  was  bought  out,  it  was  noted  that  one  of  major  problems  was  how  different  units 
work  together.  Systems  engineering  began  an  effort  to  document  processes  at  the  organiza¬ 
tion  level  to  resolve  this  problem. 

Hypothesis:  A  major  difference  between  low  and  high  maturity  organization  is  that  high  ma¬ 
turity  organizations  have  the  data  to  prove  and  demonstrate  that  their  improvements  are  suc¬ 
cessful. 

Since  there  is  no  mandate  to  use  CMMI,  one  reason  for  system  engineering  choosing  to  go 
ahead  with  CMMI^'’’  was  having  seen  the  success  of  software  engineering  using  SW-CMM®. 

In  another  case,  a  software  person  moved  over  to  systems  engineering  because  he  knew  the 
CMM®  and  improvement  methods  and  they  wanted  him  to  implement  the  systems  engineer¬ 
ing  processes.  In  another,  the  software  manager  was  made  equal  to  the  systems  engineering 
manager,  where  previously  software  reported  to  systems. 

Organizations  choosing  CMMI^'’^  are  making  a  strategic  decision.  The  VP  of  Engineering 
was  the  sponsor  in  one  case.  CMMI^'’’  should  bring  another  organization  closer  to  looking  at 
business  development  and  evaluation  -  so  sponsorship  may  be  at  a  higher  level. 
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What  made  the  light  go  on  among  senior  executives  and  others?  In  one  case,  some  people 
(engineers  and  leaders)  recognized  problems  in  their  own  area  and  saw  what  was  happening 
with  process  improvements  elsewhere  in  the  organization  -  their  initiative  drove  a  bottom-up 
improvement  effort.  In  another  case,  an  enlightened  customer  made  a  huge  difference  by 
driving  the  organization  to  improvement  (e.g.,  the  customer  said  that  they  thought  it  would 
take  a  high  maturity  organization  to  win  the  contract).  Engineers  and  program  managers 
have  to  keep  reminding  senior  executives  of  customer  comments  in  order  to  maintain  sup¬ 
port. 

There  have  been  few  CMMf  assessments  at  this  point.  Many  organizations  are  now  look¬ 
ing  at  CMMI,  making  improvements,  and  evaluating  internally  against  CMMI^*^.  Some  or¬ 
ganizations  are  planning  for  formal  assessments  in  a  year  or  two. 

One  organization  is  performing  a  Pilot  Assessment.  They  formed  a  steering  group  at  the  be¬ 
ginning  of  the  year  to  plan  for  the  assessment.  They  have  had  Intro  to  CMMI^'^  taught  on 
site  to  about  30  people,  then  conducted  assessment  team  training.  They  allowed  3  weeks  for 
the  assessment,  plus  another  week  for  the  training.  The  goal  is  to  evaluate  the  assessment 
method  and  not  to  focus  on  capability  levels  or  outcomes.  They  are  looking  at  22  Process 
Areas  (PAs).  They  are  performing  assessment  with  internal  people  and  providing  the  data  to 
SEI.  SEI  will  take  data  and  do  analysis  for  comparison  with  other  pilots.  A  focus  of  the  pilot 
is  trying  to  reduce  the  time  on  site  but  maintain  the  rigor  of  the  evaluation.  Also,  they  will 
have  SEI  observers  who  will  prepare  reports  during  assessment.  They  will  capture  questions 
about  the  model  as  well  as  the  assessment  method  (SCAMPI). 

Issue  E:  Selection  of  CMMI  Representation  -  Staged  or 
Continuous  ? 

There  was  a  brief  discussion  of  some  of  the  perceived  differences  between  the  staged  repre¬ 
sentation  and  the  continuous  representation  of  the  model. 

In  the  staeed  representation: 

■  The  concepts  can  be  communicated  clearly  with  senior  management 

■  There  is  an  element  of  simplicity  to  the  model  structure 

■  All  institutionalization  understanding  is  contained  to  process  areas 

■  If  an  organization  is  risk  averse  and  does  not  have  a  process  culture,  the  additional 
elaboration  may  help  them 

■  The  structure  supports  top  down  process  improvement 

In  the  continuous  representation: 

■  Material  is  parceled  into  arbitrary  levels 

■  An  organization  can  pick  and  choose  an  area  and  focus  on  the  particulars  of  that  area 

■  The  21  processes  areas  in  Level  3  may  be  overwhelming  to  new  organizations 

■  An  organization  can  assess  progress  in  specific  process  areas  that  are  chosen  for  their 
business  value 

■  An  initial  assessment  may  provide  more  granularity  in  results  to  help  in  decision-making 
afterward 
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There  was  a  discussion  of  some  of  the  organizational  and  environmental  factors  that  may 
influence  use  of  staged  vs.  continuous  representation. 

The  staged  representation  works: 

■  Best  in  an  organization  with  strong  functional  orientation 

■  In  environments  that  typically  do  not  use  Integrated  Product  Teams,  and/or  software  is 
not  team-based 

■  In  organizations  where  the  management  is  far  removed  from  engineering  (i.e.,  a  closed 
organization  that  requires  push  to  management) 

■  For  organizations  that  cannot  use  data  well,  since  too  much  data  is  reported  back  from 
the  continuous  results 

■  For  very  large  organizations  and  differentiated,  since  the  staged  results  are  more  easily 
shared  with  senior  management 

The  continuous  representation  works: 

■  For  organizations  who  may  not  have  a  real  engineering  process  estab¬ 
lished/defined/documented,  since  they  can  begin  in  designing  a  life  cycle 

■  For  organizations  who  have  sophisticated  engineering  and  products,  since  they  may  find 
the  granularity  and  incremental  change  beneficial 

■  For  organizations  that  are  team-based,  IPT-based,  and/or  management  is  very  close  to 
engineering 

■  For  examination  of  a  very  focused  area,  with  few  levels  of  differences 

Organizations  who  come  to  CMMI  and  have  never  done  CMM  will  approach  the  model  rep¬ 
resentation  selection  process  differently.  Some  organizations  may  not  see  benefits  of  the 
staged  representation,  which  software  folks  take  for  granted.  Some  organizations  and  cus¬ 
tomers  need  the  constraints  of  the  staged  representation;  while  others  find  they  cannot  stand 
staged. 

Organizations  with  experience  using  the  Systems  Engineering  CMM  and  continuous  assess¬ 
ments  with  a  Software  CMM®  experience-base  react  differently  than  organizations  where 
software  and  system  were  more  separate  and  using  different  models. 

When  doing  CMMI  and  communicating  adoption  principles  to  higher  executive  level,  this 
level  of  management  may  or  may  not  have  engineering  background,  model  knowledge,  etc. 
to  fully  appreciate  the  concepts  of  the  model  differences.  Executives  do  not  want  to  have  to 
make  decisions  about  subtlety.  If  an  organization  chooses  to  adopt  CMMI,  they  have  to  fig¬ 
ure  out  how  to  clearly  communicate  in  concepts  that  can  be  understood  by  senior  manage¬ 
ment  (i.e.,  concepts  which  are  based  on  business,  not  models.).  In  an  organization  where 
SW-CMM®  has  been  used,  senior  management  will  still  probably  be  conditioned  to  ask 
about  maturity  level  ratings. 

Even  though  there  are  multiple  representations,  it  is  not  necessary  that  an  organization  stay 
with  just  one  representation  or  methodology. 

Both  the  continuous  and  staged  representations  of  the  model  can  help  organizations  get  to 
Maturity  Level  5;  however,  the  model  does  not  help  organizations  go  beyond  Level  5.  The 
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focus  beyond  Level  5  is  uncharted  territory  for  the  model.  This  may  require  going  back  to 
TQM  roots  and  looking  at  organization  goals. 

An  organization  may  choose  to  focus  on  improvement  of  observable  behaviors  by  applying 
the  generic  practices  from  CMMI.  They  can  be  used  to  communicate  and  work  with  im¬ 
provement  in  organizations  with  a  history  of  TQM.  TQM  was  not  based  on  clearly  observ¬ 
able  behavior;  however,  CMMs  contain  only  observable  behavior. 

The  CMMI  model  needs  to  be  used  and  understood.  Selection  of  the  representation,  or  de¬ 
termining  when  to  do  what  in  terms  of  process  improvement  implementation,  is  coupled  to 
both  culture  and  perspectives  of  the  organization  and  stakeholders.  The  model  helps  because 
ah  organization  can  make  choices  even  within  the  model  for  improvement  priorities. 

Issue  F:  How  will  CMMI^*^  apply  to  commercial  or  other 
software  only  organizations? 

What  cautions  and  opportunities  does  CMMI^'^''  provide  the 
commerciai  software-only  organizations? 

One  organization  has  been  doing  improvements  based  on  CMM®  principles  across  the  entire 
company.  They  have  been  looking  at  CMMI^*^  generic  practices  and  common  process  areas 
for  the  whole  company  and  use  the  engineering  Process  Areas  to  improve  where  applicable. 
They  develop  software  only,  but  they  still  have  problems  with  product  lines  and  problems 
with  requirements.  The  differences  between  software-only  companies  and  those  working 
with  large  systems  is  one  of  scale  rather  than  of  engineering  practices.  They  haven’t  seen 
problems  with  interpretation  of  practices  -  they  have  handled  interpretations  of  terminology 
and  scaling  down  practices  to  work  in  a  small  company.  For  example,  they  use  a  general  Re¬ 
view  Board  for  requirements  control,  CCB  and  process  reviews.  They  relied  upon  a  former 
SEI  staff  member  (Suzy  Garcia)  to  help  with  interpretation  during  the  first  year  of  transition 
from  software  CMM®.  After  that,  they  did  their  own  thing.  CMMI^”^  makes  explicit  what 
they  were  doing  already  in  using  the  CMM®  principles  across  the  organization  -  i.e.,  the  ge¬ 
neric  practices. 

What  part  of  CMMI®*^  might  software-only  organizations  find  irrelevant?  Very  few  elements 
are  believed  to  be  irrelevant  to  most  organizations,  except  for  acquisition.  All  of  the  engi¬ 
neering  practices  may  be  applied  to  software  only  organizations.  All  organizations  interpret 
models  to  satisfy  business  goals  and  objectives.  Differences  in  interpretation  come  from  dif¬ 
ferent  size  organizations,  or  those  with  different  outputs  or  products.  What  process  areas  are 
more  important  in  a  particular  company?  None  of  the  practices  are  unimportant  -  some  may 
be  more  important  or  implemented  differently,  depending  on  the  context. 

Concern;  The  maturity  levels  have  gotten  very  large  (large  number  of  Process  Areas).  Is  it 
possible  to  extract  the  process  essentials  at  different  levels? 

Organizations  have  to  tailor  model  to  their  specific  context.  Determine  what  you  want  to  ac¬ 
cept  and  what  you  don’t  need.  This  is  the  key  to  making  the  CMMI*"^  work  in  different  or¬ 
ganizations.  Is  it  easy  to  tailor  this  model?  If  you  don’t  want  to  use  part  of  the  model,  you 
should  document  your  reasons  so  you  can  explain  this  to  an  evaluator  or  assessor.  With  a 
large  project  and  teams,  you  can  take  slavish  obedience  to  CMMI^”^.  Smaller  organizations 
may  need  expert  knowledge  in  tailoring  the  model.  CMMI®”^  is  larger  than  SW-CMM,  but  it 
may  have  lost  some  of  the  essentials.  There  is  a  dichotomy  between  being  lean  and  provid¬ 
ing  information  that  helps  users  and  assessors.  Everything  in  CMMI^”^  is  right  in  line  with 
what  are  called  “lean  practices”,  but  it  is  700+  pages. 
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In  97  -  high  maturity  organizations  were  concerned  they  were  losing  senior  management 
sponsorship  because  had  made  it  to  L5.  Still  true? 

These  days,  senior  management  sees  6-Sigma,  Lean  and  CMM®  as  different  initiatives, 
when  they  are  all  basically  the  same.  6-Sigma  became  an  initiative  in  TQM  and  was  recog¬ 
nized  to  be  of  value  beyond  software  and  system  engineering.  It  became  the  focus  of  all 
workforce  practices.  By  defining  defects  in  other  processes,  e.g.,  marketing,  6-Sigma  be¬ 
came  the  umbrella,  and  for  software  and  system  engineering,  another  tool  for  SPC.  At  one 
organization,  all  senior  management  are  green  belts  in  6-Sigma.  They  set  quantitative  goals 
for  their  areas.  Executives  that  came  from  different  backgrounds  and  from  different  compa¬ 
nies  are  now  all  working  together  under  this  umbrella. 

If  we  are  saying  that  the  basic  CMM®  improvement  process  can  be  tailored  for  any  environ¬ 
ment,  why  is  the  CMMI^"^  model  different?  Why  is  it  bigger?  Case  specific  tailoring  some¬ 
times  leaves  out  specific  practices.  The  assessment  time  is  longer.  It  isn’t  clear  that  a  given 
organization  needs  to  adopt  all  of  the  practices  in  CMMI^"^  and  whether  that  would  improve 
the  bottom  line. 

What  practices  in  the  CMMI®”^  are  not  applicable?  The  general  consensus  is  that  software 
organizations  will  apply  all  of  the  practices  in  CMMI^'^.  How  is  it  too  heavy?  Implementa¬ 
tion  of  CMMI^'^  should  be  focused  on  continuous  improvement  not  on  assessments. 

CMMI^”^  is  a  process  model  not  just  a  set  of  best  practices  to  evaluate  the  maturity  of  an  or¬ 
ganization. 

Do  the  engineering  PAs  add  value  to  software  only  organizations? 

The  Risk  Management  PA  will  strengthen  weak  areas  that  haven’t  been  able  to  communicate 
to  senior  management.  The  continuous  model  with  its  focus  on  continuous  improvement  is 
opening  up  areas  for  process  improvement.  At  least  one  organization  is  using  the  CMMI^'^ 
as  a  checklist  for  finding  improvement  opportunities  in  their  current  engineering  processes. 
Another  organization  adopted  the  SW-CMM®  and  SE-CMM®  with  one  single  process 
group.  They  had  been  continually  comparing  the  two  CMMs,  wanting  to  emphasize  the  simi¬ 
larities,  and  CMMI*’’’  helps  with  this.  Now  everyone  is  working  toward  one  model. 

CMMI^*^  seems  to  be  providing  a  logical  extension  to  what  many  organizations  already  had 
for  software.  In  many  ways,  it  can  be  considered  a  kind  of  a  super-set  of  the  SW-CMM®.  If 
so,  why  are  they  separate  programs  (SW-CMM®  and  CMMI),  without  a  clear  progression 
from  one  to  the  other?  Why  does  SW-CMM®  have  to  be  “ended”?  Why  isn’t  there  a  logical 
progression  from  software  CMM®  to  CMMI®^  -  training,  assessment,  everything?  We  need 
to  ask  SEI  this  question. 

Issue  G:  Do  Level  5  organizations  develop  very  different 

aiternative  practices  ?  Are  there  differences  based  on 
organizational  structures  (e.g.,  hierarchical  vs. 
flatter)? 

This  issue  was  raised  in  the  working  group;  however,  was  not  discussed  during  the  working 
group  session. 

Recommendations  for  High  Maturity  Organizations 

Recommendations 
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The  group  discussion  expanded  to  cover  wider  maturity  with  respect  to  CMMI^"^  adoption 
rather  than  just  higher  maturity.  Much  of  the  discussion  centered  on  strategic  issues  and 
business  decisions  of  model  selection.  It  was  noted  that  offers  the  wider  maturity 

option  and  a  broader  opportunity  for  integration  of  disciplines. 

As  a  result  of  the  working  group’s  discussions,  the  following  recommendations  for  organiza¬ 
tions  seeking  to  achieve  high  maturity  were  formulated: 

Recommendation  HM-1  ~  High  maturity  software  organizations  have  some  valuable  lessons 
learned  that  other  organizations  can  use  in  advancing  through  the  maturity  levels  and  as  other 
organizations  mature  through  Implement  measurements  early,  set  up  an  engineer¬ 

ing  process  group,  peer  reviews  and  other  forms  of  verification,  process  improvement  adop¬ 
tion  lessons  learned. 

Recommendation  HM-2  --  There  are  not  huge  differences  between  CMMI^*^  and  SW 
CMM®,  so  that  is  comforting.  Follow  Total  Quality  Management  (TQM)  principles  during 
strategic  planning;  identify  marketing  areas  and  operational  direction.  If  you  have  an  initia¬ 
tive  that  crosses  the  organization  (i.e.,  establishes  an  “umbrella”),  it  becomes  easier  to  deploy 
CMMI^*^  due  to  a  common  framework. 

Recommendation  HM-3  ~  Industry  has  to  examine  territory  beyond  maturity  Level  5. 

Observations 

In  addition  to  the  observations  above,  the  working  group  identified  a  number  of  other  obser¬ 
vations  relevant  to  high  maturity  organizations.  These  were: 

■  Level  of  impact  and  effort  in  a  high  maturity  organization  should  be  minimal  due  to 
natural  extension  from  SW  CMM®  to  CMMI®”^. 

■  Selection  of  the  Staged  versus  Continuous  implementation  may  depend  on  some 
cultural,  environmental,  and  management  factors. 

■  CMMI^^  can  be  especially  beneficial  to  organizations  with  less  mature  Systems  Engi¬ 
neering  groups. 

■  CMMI®*^  provides  commonality  in  process  improvement  across  Software  and  Systems 
engineering  disciplines. 

■  CMMI®*^  assessments,  formal  and/or  less  formal,  can  be  used  to  assess  the  feasibility  of 
application  of  the  CMMI*"^  practices  and  assessment  method  for  an  organization. 

■  CMMI^”^  generic  practices  can  be  successfully  applied  to  non-engineering  business  ar¬ 
eas  to  support  process  improvement. 

■  Basically  CMMI^*^  has  broadened  the  base.  Implementation  has  more  to  do  with  size  of 
the  organization  than  with  disciplines  in  the  organization. 

It  was  also  noted  that  the  cost  for  process  assessments  is  very  high,  and  that  the  SEI  needs  to 
provide  a  less  expensive  and  time-consuming  assessment  method  for  CMMI^"^.  The  three 
classes  of  planned  CMMI^”^  assessments  were  briefly  discussed.  Class  A  assessments  reflect 
the  rigorous  process  used  in  order  to  achieve  ratings  and  proclaim  results  to  the  world.  The 
Class  B  and  C  assessments  are  designed  to  be  more  lightweight  methods  that  cost  less  but  are 
a  quick  check  of  where  an  organization  stands  against  the  model. 

Recommendations  for  the  SEI 

As  a  result  of  the  working  group's  discussions,  the  following  recommendations  for  the  SEI 
were  formulated: 
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Recommendation  SEM  -  Why  is  CMMI^^^  not  considered  SW  CMM®  Version  3.0?  Why 
is  there  not  a  logical  progression  from  SW-CMM®  to  (in  models,  training,  and 

assessment  methods)?  In  lieu  of  such  a  progression,  organizations  will  have  a  more  complex 
transition. 

Recommendation  SEI-2  -  Develop  a  time -bound  release  plan  for  industry  -  involv¬ 

ing  all  aspects  of  the  organizations  (e.g.,  marketing.  Human  Resources,  etc.)  Take  an  enter¬ 
prise-wide  assessment  approach,  e.g.  Malcolm  Baldrige. 

Recommendation  SEI-3  —  High  maturity  organizations  have  learned  how  to  quickly  and  in¬ 
telligently  implement  continuous  process  improvement.  Capture  those  lessons  learned  for  the 
sake  of  others  and  provide  industry  a  road  map  to  get  through  the  model.  That  information 
could  be  used  to  fine-tune  the  model,  (e.g.  case  studies. . .) 

Recommendation  SEI-4--  CMM®  has  established  itself  as  an  international  de  facto  standard. 
It  is  not  desirable  to  risk  that  investment  by  a  badly  managed  transition  to  CMMI,  such  that 
the  user  community  loses  faith  in  CMMs.  Defense/aerospace  industry  community  alone  can¬ 
not  keep  CMMI^^  alive  and  surviving,  it  has  to  be  accepted  around  the  world.  Establish  in¬ 
dustry-wide  support  and  buy-in,  including  involvement  from  the  commercial  sector. 

CMMI^^  has  been  focused  on  too  narrow  of  a  world  (initially).  Ensure  that  software-only 
organizations  can  see  that  the  model  works  for  them  too  and  that  there  are  clear  guidelines  of 
the  model  for  application  to  software-only  organizations. 
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•  Context 

The  author  of  this  paper  is  a  member  of  the  CMMI  product  team  and  has 
experience  of  CMMI  training  and  assessments  both  in  Europe  and  the 
USA. 


•  Barriers  to  the  uptake  of  the  CMMI 

Before  discussing  adoption  mechanisms,  it  may  be  instructive  to  analyze 
this  new  technology  using  Rogers  (Rogers,  Everett  M.,  Diffusion  of 
Innovations,  fourth  edition.  Free  Press,  New  York,  1995)  theories  on  the 
diffusion  of  innovation.  There  are  several  characteristics  of  a  technology 
change  or  innovation  diffusion  situation  that  influence  the  uptake  of  the 
new  technology.  The  first  is  the  innovation  it  self. 

•  Relative  advantage.  Although  the  any  of  the  CMMI  models  should 
have  a  high  degree  of  relative  advantage  over  the  prevalent  source 
models,  due  to  the  potential  cost  savings  with  single  assessment, 
single  training,  integrated  improvement,  etc.  This  advantage  is 
drowning  in  noise  and  misunderstanding.  Furthermore  that  is  an 
expectation  that  the  CMMI  is  an  improvement  on  the  SW-CMM  - 
new  and  better.  The  world  at  large  was  not  expecting  a  model  from 
the  SEI  that  was  not  perfect  on  day  1. 

•  Compatibility.  Since  the  flavor  of  both  the  SW-CMM  and  EIA  731 
models  is  discernable  in  the  CMMI,  it  would  be  logical  to  think 
that  there  is  a  high  degree  of  compatibility  between  the  new  model 
and  its  predecessor.  Potential  adopters  approach  the  CMMI  with 
this  expectation  and  find  instead  familiar  concepts  or  terminology 
with  a  different  twist,  or  additions  from  the  other  source  models 
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that  are  then  perceived  to  be  superfluous,  and  difficult  to  accept. 
This  reduces  the  perceived  degree  of  compatibility 

.  Complexity.  For  the  practitioners  and  managers  in  the  trenches 
the  abstract  concepts  of  process  and  process  management  can  be 
complex  enough.  With  more  than  one  model  and  two  architectures, 
the  CMMI  has  a  very  high  degree  of  complexity.  Typographical  er¬ 
rors  tend  to  increase  the  complexity  as  well  as  detract  from  the 
credibility  of  the  model.  A  number  of  potential  adopters  attempt  to 
understand  the  CMMI  by  “browsing”  through  the  documentation. 
In  a  very  natural  attempt  to  reduce  the  complexity,  model  ele¬ 
ments  are  ignored  and  the  model  is  reduced  to  the  adopters  “com¬ 
fort  zone”.  The  adopter’s  conclusion  then  is  that  the  model  contains 
too  much  superfluous  information  making  use  cumbersome. 

•  Trialability.  The  intended  use  of  the  CMMI  is  for  process  im¬ 
provement,  which  would  be  manifested  in  a  CMMI-based  im¬ 
provement  program  or  a  CMMI-assessment.  Industry  working  un¬ 
der  time-to-market  pressures  will  not  find  it  feasible  to  allocate  the 
extra  effort  to  pilot  the  CMMI,  unless  the  decision  to  transition  has 
been  made.  Therefore  the  most  feasible  method  of  investigating  the 
CMMI  is  by  attending  seminars,  tutorials,  presentations  or  train¬ 
ing.  The  conclusions  that  potential  adopters  reach  is  therefore  con¬ 
tingent  on  the  perceived  message  delivered  both  by  the  present¬ 
ers/trainers  and  by  other  participants  in  the  event. 

•  Observerbility.  Potential  adopters  are  very  interested  in  hearing 
about  other’s  experiences  with  the  CMMI.  The  well-attended 
CMMI  tracks  at  the  recent  SEPG  is  an  example  of  this. 

Other  relevant  characteristics  of  the  diffusion  are  the  communication 
channels  and  the  decision  process.  In  the  software  industry  decisions  to 
transition  to  new  technology  appear  to  be  influenced  either  by  a  prepon¬ 
derance  of  anecdotal  or  real  evidence  that  a  new  technology  is  beneficial, 
or  a  customer  or  legal  requirement.  Many  industrial  organizations  rely 
either  directly  or  indirectly  on  advise  from  trusted  sources,  such  as  con¬ 
sultants.  The  message  regarding  the  CMMI  from  these  sources  is  mixed. 
The  reason  for  this  mixed  messagee  is  that  consultants  themselves  are 
grappling  with  understanding  the  new  model  as  well  as  accepting  the  fact 
that  even  change  agents  may  need  to  change. 

Some  anecdotal  evidence  of  customers  requiring  use  of  the  SW-CMM  or 
CMMI  is  beginning  to  be  heard,  but  this  is  fairly  weak.  Given  that  the 
model  is  not  yet  a  year  old  there  is  no  real  or  evidence  of  business  benefits 
from  using  the  CMMI.  Since  there  is  ROI  evidence  from  use  of  the  SW- 
CMM,  a  possible  inference  is  that  a  CMM-model  with  a  larger  scope  - 
both  systems  and  software  -  would  yield  larger  ROI.  Some  accept  this  in¬ 
ference  and  some  don’t.  Furthermore  there  is  no  reliable  information 
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about  which  companies  are  have  made  the  decision  to  or  are  even  consid¬ 
ering  transitioning  to  the  CMMI. 


1  Speeding  the  adoption  of  the  CMMI 

1.1  What’s  needed 

As  the  CMMI  custodians  the  source  of  the  message  is  the  SEI.  Other 
messengers  come  to  the  SEI  to  learn.  As  the  distance  between  the  source 
and  the  receiver  increase  the  message  gets  weaker  and  diluted.  Therefore 
a  strong,  consistent,  clear  and  firm  message  from  the  “source”  is  needed. 
There  should  be  an  agreed  upon  interpretation  of  all  elements  of  the 
model.  This  interpretation  is  what  should  be  given  at  all  training,  semi¬ 
nars,  presentations  etc.  (Since  the  number  of  people  involved  in  develop¬ 
ing  the  model  was  large  this  in  itself  is  not  an  easy  task.)  Typographical 
and  other  minor  glitches  must  be  removed. 

The  presentation  of  the  two  representations  of  the  CMMI  as  different 
animals  causes  problems  of  imderstanding  and  acceptance  as  well  as  the 
death  of  many  trees.  In  essence  the  two  representations  are  just  different 
logical  views  of  the  same  data.  A  better  presentation  would  be  to  present 
the  model  elements  (process  areas  and  practices)  separately  and  then  add 
a  couple  of  chapters,  one  discussing  the  logical  view  called  “staged”  and 
the  other  discussing  the  logical  view  called  “continuous”. 

Provide  discipline  specific  seminars.  Users  of  one  of  the  source  models 
need  to  understand  the  domains  of  the  additional  disciplines.  This  will  fa¬ 
cilitate  the  acceptance  of  the  various  model  elements  and  hopefully  will 
contribute  to  a  widening  of  the  users  comfort  zone. 

Provide  incentives  to  those  who  will  carry  the  message  forward.  A  sim¬ 
ple  form  of  incentives  is  to  lower  license  fees  and  training  costs.  Other 
more  sophisticated  (or  brutal?)  forms  could  be  yearly  increases  of  licenses 
fees  for  the  source  models,  to  make  it  economically  sensible  to  transition. 
CMMI-related  training  such  as  domain  specific  seminars  or  interpretation 
seminars  could  be  distributed  via  CD  or  the  web  for  free.  Licensed  asses¬ 
sors  could  be  provided  with  supporting  tools. 

Make  evidence  of  the  model-uptake  available  by  publishing  a  list  of 
those  companies  who:  are  investigating  the  CMMI,  and  those  who  have 
decided  to  transition  to  the  model.  CMMI  marketers  need  to  be  able  to 
reference  a  reliable  source  of  uptake  evidence. 


1.2  What’s  new 
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This  sections  describes  a  couple  of  specific  “tools”  used  when  working  with 
the  CMMI. 

The  first  tool  is  a  SW-CMM  tool  converted  for  use  with  the  CMMI.  The 
tool  called  CMMIOnBoard  is  a  simple  self-appraisal  tool  coupled  with  an 
improvement  engine  (See  Appendix  1  for  a  pictorial  representation).  The 
original  version  of  the  tool  is  implemented  in  Microsoft  Excel,  though  at¬ 
tempts  have  been  made  to  achieve  the  same  functionality  in  Microsoft 
PowerPoint.  The  “Boarding  sheets”  -  essentially  one  per  Process  Area 
contain  the  both  the  generic  and  specific  practices  of  the  CMMI.  The 
sheets  or  boards  are  the  core  of  the  OnBoard  process,  which  has  five  ac¬ 
tivities. 

1.  Initial  Boarding.  Appropriate  groups  use  the  onBoard  sheets  to  ap¬ 
praise  themselves  against  the  practices  of  individual  process  areas. 
The  appropriate  group  is  dependant  on  the  organization’s  im¬ 
provement  goals.  Observations  are  written  on  colored  post-its  which 
also  show  degree  of  fulfillment,  red  -  weakness,  yellow  -  partially 
fulfilled,  green  -  fulfilled. 

2.  Action  Planning.  Yellow  and  red  stickers  are  input  to  this  activity. 
The  actions  are  prioritized  and  improvement  begins. 

3.  Board  Updates.  Approximately  once  a  month  the  boards  are  up¬ 
dated  by  the  group  that  did  the  initial  boarding  or  a  subset  of  that 
group.  New  observations  are  written  on  post-its  and  placed  on  the 
original  stickers  in  such  a  way  that  all  colors  are  visible.  The  intent 
is  to  make  improvement  progress  visible. 

4.  Board  Walk.  Approximately  once  a  quarter  the  person  responsible 
for  the  board  (project  manager,  process  owner,  EPG  chairman)  pre¬ 
sents  the  boards  to  the  senior  manager.  Issues  that  are  slowing 
process  improvement  are  discussed.  Issues  common  across  projects 
or  process  areas  are  identified  and  planned  for. 

5.  Board  Checks.  This  activity  is  essentially  a  light  or  informal  as¬ 
sessment.  The  boards  are  input  to  the  assessment,  which  is  ex¬ 
pected  to  occur  every  six  months. 

Generally  the  boards  are  kept  in  a  central  place,  so  improvement  progress 
is  visible,  though  sometimes  company  culture  or  logistics  does  not  allow 
this.  The  tool  has  worked  well  for  organizations  using  the  SW-CMM.  The 
CMMI  version  exists  for  both  the  continuous  and  staged  representations. 
Activities  1  and  2  have  been  piloted  with  out  problems.  The  improvement 
engine  has  yet  to  be  piloted. 

The  second  tool  is  actually  a  concept  has  been  developed  specifically  for  an 
organization  that  has  chosen  to  transition  to  the  CMMI  continuous  repre¬ 
sentation.  The  intention  is  to  integrate  improvement  planning  with  busi¬ 
ness  goals  and  to  provide  support  for  choosing  which  process  areas  to  fo- 
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cus  on.  The  concept  called  Organizational  Performance  Management 

(0PM)  has  three  main  steps: 

1.  Goal  setting.  This  activity  is  the  responsibility  of  senior  manage¬ 
ment.  It  should  occur  at  least  once  a  year,  but  could  happen  more 
often.  Goal  setting  consists  of  establishing  a  wanted  position,  iden¬ 
tifying  key  success  factors,  defining  measurable  objectives  and  key 
performance  indicators  and  lastly  developing  measurement  and 
date  definitions.  Coincidently  this  set  of  activities  would  also  fulfill 
specific  goal  1  of  the  Measurement  and  Analysis  (M&A)  process 
area. 

2.  Action  Definition.  Two  instruments  are  used  to  guide  action  plan¬ 
ning.  The  first  Gap  Analysis  compares  the  defined  measurable  ob¬ 
jectives  and  performance  indicators  with  achieved  values.  The  sec¬ 
ond  is  Cause-Effect  Analysis.  A  set  of  cause-effect  chains  have  been 
developed  as  guidance.  It  is  possible  to  map  many  causes  or  effects 
to  process  areas  and  even  specific  practices.  The  results  of  these 
analysis  are  input  to  action/improvement  planning.  Note  that  all 
causes  do  not  map  to  the  CMMI,  some  could  be  mapped  to  P-CMM, 
leading  to  a  need  to  have  people  issues  integrated  into  the  CMMI. 

3.  Data  Collection  and  Analysis.  The  third  activity  of  0PM  would  ful¬ 
fill  specific  goal  2  of  M&A. 

The  benefit  of  0PM  is  that  the  organization  clearly  sees  the  link  to  results 
and  to  business  goals.  Thus  increasing  the  buy-in  for  process  improve¬ 
ment  and  a  reduction  of  the  “process  for  process  sake”  sjmdrome.  These 
concepts  are  being  piloted  at  the  moment. 


•  Conclusion 

Organizations  that  do  not  adapt  to  changing  market  needs  do  not  survive. 
Senior  managers  and  change  agents  know  this  to  be  a  truism.  At  the  or¬ 
ganizations  level  the  question  is  what  to  change  and  when  to  do  it.  But 
organizations  are  made  up  of  people  and  therefore  change  happens  one 
person  at  a  time.  The  road  to  CMMI  must  manage  this  dichotomy. 


•  Appendix  1 
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The  purpose  of  this  presentation  is  to  lay  the  foundation  for  thinking  about  the 
various  things  that  the  SEI  and  the  community  must  produce  in  order  to  facili¬ 
tate  the  deployment  and  use  of  CMMI. 

Experience  has  shown  that  having  a  better  solution  is  no  guarantee  of  success 
in  the  marketplace  (e.g.,  Sony’s  Beta  VCR  technology,  which  captured  the 
commercial  television  market,  failed  to  capture  the  lucrative  home  market  due 
to  non- technical  issues.)  Who  you  partner  with  and  what  additional  items  are 
made  available  are  often  more  important  than  the  technical  features  of  the 
product  itself. 

To  facilitate  preparations  for  deployment  of  CMMI,  we  wish  to  leverage  ex¬ 
periences  of  others  and  consider  as  many  of  these  other  factors  as  possible  be¬ 
fore  we  commit  ourselves  to  any  specific  action  plan. 

This  presentation  introduces  a  number  of  critical  aspects  that  have  been  shown 
to  be  relevant  in  other  deployment  efforts.  We  believe  that  by  having  a  shared 
vocabulary  and  shared  mental  models  before  beginning  our  deliberations,  we 
will  increase  our  ability  to  move  toward  an  appropriate  plan  for  moving  CMMI 
from  the  SEI  into  broad  popular  usage. 
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Redwine  and  Riddle  studied  the  length  of  time  it  has  taken  to  move  proven  software 
technologies  from  the  first  relevant  paper  to  broad  popular  usage.  They  concluded 
that  it  took  from  12  to  18  years  for  those  technologies  they  studied. 

One  can  raise  all  sorts  of  arguments  about  this  study,  from  questioning  their  research 
method  and  analysis  methods  to  the  fact  that  our  world  is  very  different  today.  Even 
if  we  assume  that  some  or  even  all  of  these  points  are  true,  name  one  software  inno¬ 
vation  of  any  real  consequence  that  has  taken  less  than  12  years  to  move  from  first 
seminal  paper  to  broad  popular  usage! 

In  a  field  where  hardware  generations  are  measured  in  months  (from  18  to  24,  de¬ 
pending  on  your  source),  12  years  is  six  generations  and  this  leads  us  to  wonder 
what  potentials  are  being  missed. 

It  has  been  stated  that  the  full  power  of  the  Pentium  IV  will  not  be  realized  for  years, 
as  none  of  Microsoft’s  compilers  are  optimized  to  generate  code  that  runs  well  on  the 
chip.  It  will  be  a  year  or  so  until  new  compilers  exist  that  are  optimized  for  the  Pen¬ 
tium  IV.  It  will  take  another  year  or  so  for  these  compilers  to  make  their  way  into  the 
hands  and  computers  of  software  developers  and  longer  for  them  to  learn  to  write 
code  in  ways  that  take  full  advantage  of  the  chip’s  features.  The  operating  systems 
and  most  applications  will  have  to  be  rewritten,  not  just  recompiled,  for  the  full  po¬ 
tential  of  the  Pentium  IV  to  be  realized  by  the  typical  computer  user.  Some  have 
suggested  that  most  of  the  software  running  on  PCs  today  are  really  optimized  for 
CPUs  that  are  no  longer  available. 
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Adoption  often  takes  too  long 

For  all  of  our  hard  work,  the  gap  between  common 
practice  and  best  practice  is 

•  Still  large 

•  And  some  even  think  its  growing 

Efforts  to  adopt  new  technology  to  close  the  gap  are 

•  Costly 

•  Unpredictable 

•  Easily  lost  through  reorganization,  mergers,  and 
turnover  of  personnel 
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There  are  people  in  the  software  development  business  who  know  how  to  create 
software  that  works  well.  American  telephones  have  been  taking  advantage  of  com¬ 
puter-based  switches  for  several  decades. 


How  often  have  you  picked  up  a  telephone  handset  on  a  wired  telephone  and  failed  to 
connect  a  phone  call  due  to  software  failure  in  a  telephone  company  switch?  When 
was  the  last  time  you  had  to  reboot  your  PC  or  had  an  application  freeze?  The  size 
and  complexity  of  the  software  in  a  telephone  switch  is  far  larger  than  anything  in  a 
typical  PC. 

Data  shows  that  the  total  lifecycle  costs  associated  with  producing  software  “quick 
and  dirty”  are  far  higher  than  the  costs  associated  with  producing  high  quality  soft¬ 
ware.  Unfortunately,  many  people  believe  that  being  first  in  the  market  with  a  low 
quality  product  is  worth  much  more  than  the  P&L  numbers  would  indicate.  This  lack 
of  discipline  and  sound  business  fundamentals  have  resulted  in  a  number  of  “dot 
gones”. 

At  a  time  of  growing  application  size  and  complexity,  the  number  of  computing 
graduates  from  recognized  schools  as  a  fraction  of  new  jobs  being  filled  is  falling. 
Too  many  people  believe  that  one’s  ability  to  write  code  is  more  important  than  any¬ 
thing  else,  even  when  study  after  study  shows  that  software  developers  spend  much 
less  than  50%  of  their  time  writing  code.  Convincing  developers  to  use  new  tech¬ 
nologies  when  they  lack  the  background  to  understand  them  is  a  real  up-hill  battle. 
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One  of  the  promises  of  the  CMM  was  the  notion  that  software  development  would 
become  less  unpredictable  and  less  expensive.  A  number  of  studies  from  a  number  of 
organizations  support  this  belief. 

Another  interesting  fact  is  that  the  disciplines  of  the  CMM  actually  provide  other 
benefits  to  the  organization.  Moving  from  CMM  Level  One  to  Level  Two  takes,  on 
average,  25.5  months.  The  change  required  is  limited  to  six  key  process  areas  and 
most  of  the  change  is  limited  to  project  leaders.  The  change  from  Level  Two  to 

Level  Three  requires  addressing  seven  key  process  areas,  requires  revamping  a  great 
deal  of  the  work  required  to  get  to  Level  Two,  requires  change  in  most  all  of  the  de¬ 
velopment  organization,  and  yet  the  average  time  to  move  to  Level  Three  is  only  19 
months. 

Examining  the  graphics  above  also  shows  that  the  variation  from  smallest  non-outlier 
to  largest  non-outlier  has  been  reduced  and  the  variation  from  the  25th  percentile  to 
the  75th  percentile  has  similarly  reduced. 

Any  way  you  look  at  it,  these  organizations  have  made  more  and  more  complex 
changes  in  less  time  and  with  less  unpredictability. 
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You  must  change  all  three 


The  means  by  which  people,  procedures,  methods,  equipment, 
and  tools  are  integrated  to  produce  a  desired  end  result. 
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If  you  are  going  to  change  how  work  gets  done,  you  must  change  three  critical  as¬ 
pects  of  the  organization: 

1.  The  people.  If  the  people’s  behaviors  don’t  change,  nothing  will  really  be 
different. 

2.  The  Procedures  and  Methods.  Changes  of  behaviors  are  important,  but  if 
those  behaviors  are  not  captured  in  procedures  and  methods,  different  bright 
people’s  interpretations  of  the  change  is  likely  to  collide  with  those  of  others, 
resulting  in  waste. 

3.  Tools.  There  are  some  things  that  people  with  procedures  and  methods  can  do 
well  and  there  are  some  things  that  require  tools.  Without  tools,  the  full  value 
of  a  change  is  not  likely  to  be  realized. 

You  must  also  change  how  these  three  critical  elements  interact.  Process  is  how  a 
group  of  people  organize  themselves,  how  work  flows  between  them,  and  how  les¬ 
sons  and  wisdom  from  the  past  influences  work.  Without  good  processes,  critical 
lessons  tend  to  be  ignored,  work  handoff  is  not  smooth,  time  is  wasted,  work  is  dupli¬ 
cated  or  not  done  at  all,  and  there  is  no  vehicle  for  things  to  improve. 

As  the  designer  of  a  new  product,  we  must  address  these  three  critical  aspects  of  work 
and  how  they  are  integrated  into  a  total  workflow  if  we  are  to  realize  the  full  benefit 
of  the  technology. 
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A  good  first  step  is  to  organize  all  of  the  information  at  our  disposal.  With  so  many 
lessons  and  ideas  out  there,  it  is  easy  to  become  overwhelmed  and  ignore  something 
important. 

The  framework  provided  above  is  an  early  attempt  to  spread  out  the  problem  space  in 
order  to  make  it  easy  to  find  information  relevant  to  the  problem  at  hand. 

The  domains  across  the  top  of  the  framework  separate  out  the  various  areas  of  focus, 
while  the  phases  down  the  left  side  spread  out  the  various  steps  in  the  life  cycle  of  a 
major  change. 

Experience  will  teach  which  domains  are  most  critical  to  a  specific  firm  as  it  goes 
from  wrestling  with  whether  or  not  a  change  should  be  implemented,  through  the  ac¬ 
tivities  of  properly  evaluating  the  motivations,  options,  benefits,  and  costs,  through 
adaptation  and  piloting,  to  implementation  and  learning. 

It  has  been  argued  that  Silicon  Valley’s  approach  to  new  technology  adoption  is  to 
have  companies  go  out  of  business  to  be  replaced  by  new  companies  that  employ  the 
new  methods  and  tools. 

We  believe  that  the  societal  costs  of  such  an  approach  is  far  too  high  and  is  not  an 
option  for  the  government  or  the  military.  We  believe  that  a  more  disciplined  ap¬ 
proach  can  result  in  firms  that  regularly  refresh  and  reinvent  themselves  without  dis¬ 
carding  the  bulk  of  their  workforce  or  walking  way  from  their  commitments  to  their 
customers  and  stockholders. 
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Mintzberg’s  book  on  the  Rise  and  Fall  of  Strategic  Planning  paints  a  particularly 
bleak  view  of  corporate  efforts  to  plan  their  futures.  We  believe  that  Mintzberg  has 
properly  recognized  that  many  such  efforts  fail,  but  he  has  elected  to  ignore  and 
failed  to  understand  those  cases  where  such  strategic  efforts  have  succeeded. 

Rummler  &  Brache’s  Improving  Performance  (How  to  Manage  the  White  Space  on 
the  Organization  Chart)  has  a  much  better  balance.  These  authors  recognize  that  this 
is  hard  work  and  that  many  people  have  failed  to  get  it  right.  On  the  other  hand,  they 
have  provided  solid  advice  from  organizations  that  have  succeeded  in  efforts  to  im¬ 
prove. 

One  of  the  most  important  points  is  that  the  implementation  is  just  as  important  as  the 
plan.  If  the  implementation  is  not  being  led  by  the  leader  and  the  bulk  of  the  imple¬ 
mentation  is  not  being  performed  by  the  very  best  people  in  the  organization,  the 
change  is  not  likely  to  go  well.  Leaders  have  to  choose  where  to  expend  their  re¬ 
sources.  If  they  invest  all  of  their  money  and  hot  talent  chasing  today’s  business, 
there  will  be  nothing  available  to  develop  new  business  and  prepare  the  organization 
to  execute  that  new  business.  Similarly,  not  all  of  the  money  and  talent  can  be  spent 
on  the  future  with  no  concern  about  honoring  existing  commitment  and  business 
needs. 

Rather  than  reinvent  what  business  leaders  have  learned  and  others  have  painfully 
relearned,  possibly  at  the  cost  of  losing  business,  why  not  leverage  that  knowledge 
and  skill? 
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An  approach  for  the  details 


Learning 
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Many  might  wonder  why  the  SEI  felt  the  need  to  reinvent  what  Deming  calls  the  She- 
whart  Cycle  (but  what  many  call  the  Deming  Cycle.)  What’s  wrong  with  Plan,  Do, 
Check,  Act? 

The  purpose  of  a  model  is  not  to  accurately  represent  the  real  world.  Rather,  the 
point  of  a  model  is  to  focus  attention  on  aspects  of  reality  that  are  important  and 
minimize  attention  on  those  aspects  of  reality  that  add  no  value.  While  the  basic 
Shewhart  cycle  is  fine  for  expressing  the  fundamentals,  it  fails  to  highlight  those  as¬ 
pects  of  reality  that  often  lead  to  the  failure  of  continuous  improvement.  One  key 
feature  of  the  IDEAL  model  is  the  role  of  leadership  that  is  not  present  in  the  She¬ 
whart  Cycle.  Without  clear  and  unambiguous  leadership,  how  can  the  diverse  people 
playing  a  wide  variety  of  roles  come  to  compatible  new  ways  of  working? 

Consider  the  words  of  General  John  Jumper,  former  commander  of  Air  Combat  Com¬ 
mand  and  now  the  Air  Force  Chief  of  Staff.  While  at  ACC,  he  wrote  14  main  tenants 
that  he  shared  with  his  troops,  which  include: 

We  once  had  a  quality  Air  Force  that  was  ruined  by  a  program  called  Quality  Air 
Force. 

The  product  is  more  important  than  the  process. 

I  can  read  each  of  these  statements  two  ways.  I  agree  that  QAF  was  not  implemented 
well  and  an  inappropriate  focus  on  process  at  the  expense  of  product  is  wrong.  Is  the 
General  opposed  to  continuous  improvement  or  removing  process  problems  from 
work  flows?  We’ve  not  seen  enough  elaboration  to  know.  Improvement  requires 
continuous  and  clear  leadership  from  the  top.  Juran  says  it  can’t  be  delegated. 
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Carnegie  Melioo  UirversiV 

Software  Englneerirrg  Institute 


IDEAL:  Initiating 


Sponsorship  development 

•  Understand  the  compelling  need 

•  Build  shared  vision  on  how  improvement  will  occur 

•  Establish  solid  data  on  adoption  cost  and  impact 

•  Develop  understanding  of  why  it  will  take  so  long 


Sponsorship  sustainment 

•  Support  the  sponsors  to  honor  their  roles 

•  Keep  the  sponsors  proactive  as  opposed  to  reactive 

•  Identify  measurable  milestones  along  the  way 

•  Remind  the  sponsor  why  it  takes  so  long 

•  Use  data  to  regularly  show  progress 
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One  of  the  best  continuous  improvement  efforts  I  have  witnessed  was  one  run  by  a 
Colonel  who  knew  that  the  future  of  his  software  development  organization  would 
depend  on  how  well  his  people  could  improve.  The  reputation  of  the  organization 
when  he  arrived  was  not  good  and  people  from  the  command  were  constantly  looking 
for  ways  to  leave  it  for  almost  any  other  job  on  the  base,  if  not  elsewhere  in  the  Air 
Force. 

Within  a  year  of  his  arrival,  he  had  a  larger  percentage  of  personnel  assigned  to  proc¬ 
ess  improvement  than  any  organization  I  had  ever  visited,  and  these  people  were  not 
people  with  nothing  to  do.  Some  of  the  very  best  and  most  experienced  people  had 
been  assigned  to  the  effort. 

When  asked  how  he  could  afford  to  make  such  a  large  investment  in  process  im¬ 
provement,  he  laughed  and  explained  that  one  of  the  joys  of  working  for  a  Level  One 
Command  is  that  there’s  no  way  they  could  tell  that  he  wasn’t  using  every  person  on 
development. 


He  then  stated  more  seriously  that  there’s  so  much  room  for  improvement  that  the 
benefits  of  the  investment  would  soon  exceed  the  costs.  He  also  pointed  out  that  a 
large  headcount  cut  was  coming  and  the  only  way  he  figured  he  could  survive  that  cut 
was  to  get  better  as  fast  as  possible,  and  they  better  get  used  to  a  smaller  team. 

He  led  the  continuous  improvement.  They  improved  dramatically.  Their  reputation 
turned  around.  They  were  cranking  out  more  and  better  software  with  over  30% 
fewer  people.  Leadership  and  the  role  they  play  is  paramount! 
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^rtwnre  Engineering  Institute 

IDEAL:  Diagnosing 

Informal  assessments 

•  Business  results  indicate  where  problems  might  be 

•  Project  postmortems 

•  Risk  management  activities 

•  TQM  activities 

•  Employee  suggestion  program 
Formal  assessments 

•  CMMI-based  assessment  for  Internal  improvement 

•  CMMI-based  evaluation  for  contractor  selection 
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By  including  the  Initiating  Phase,  the  IDEAL  model  draws  attention  to  the  impor¬ 
tance  of  obtaining  and  sustaining  active  and  informed  leadership.  Similarly,  the  at¬ 
tention  to  the  Diagnosing  Phase  highlights  something  under  the  surface  in  the  She- 
whart  Cycle. 

Hidden  in  Plan,  Do,  Check,  Act  is  the  notion  of  understanding  what  is  working  and 
what  isn’t.  Without  exposing  the  notion  of  a  more  formal  approach  to  diagnosis, 
people  using  the  Shewhart  Cycle  can  be  chaotic  in  how  they  plan  their  next  iteration. 
In  fact,  some  of  us  believe  that  true  Level  Five  organizations  obtain  predictability  in 
technology  and  process  change  management  by  employing  some  form  of  statistical 
control. 


The  point  of  the  diagnosis  phase  of  the  IDEAL  Cycle  is  to  focus  attention  that  one 
needs  a  good  diagnosis  before  getting  too  involved  with  planning  the  next  improve¬ 
ment.  It  is  the  SEI’s  position  that  a  good  underlying  reference  model  is  an  important 
tool  in  performing  such  a  diagnosis. 

Part  of  the  diagnosis  should  be  a  reflection  on  the  past  and  the  lessons  that  have  been 
learned  from  previous  efforts. 
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Camegie  Mellon  Univerifly 

Software  Engineering  Institute 

IDEAL:  Enablir^ 

Technology  evaluation 

•  How  might  technologies  address  the  issues? 
Solution  tailoring 

•  What  adjustments  are  needed  to  improve  the  fit? 
Usage  design 

•  How  will  the  work  flow  improve? 

•  What  roles  will  have  to  change? 

•  What  new  skills  will  be  needed? 

Adoption  planning 

•  Which  people  will  have  to  change? 

•  What  change  will  they  have  to  make? 

•  How  will  we  help  people  though  their  resistance? 

•  How  will  people  acquire  their  new  skills? 
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The  Enabling  Phase  is  about  producing  an  actionable  plan  that  has  the  highest  prob¬ 
ability  of  addressing  the  issues  brought  to  the  surface  during  the  diagnosing  phase. 
Improvement  is  usually  coupled  with  some  change  in  tool,  method,  procedure,  or  the 
process  by  which  people,  procedures,  and  tools  are  integrated  to  accomplish  the 
work. 


Evaluating  the  options,  tailoring  the  solution,  and  checking  the  usage  design  on  the 
real  work  the  workers  at  this  place  must  perform  is  important.  Few  “one  size  fits  all” 
solutions  are  worth  your  time. 

Rolling  out  the  improvement  means  changing  the  way  a  number  of  people  do  their 
jobs  while  they  are  doing  them.  This  is  like  repairing  an  airplane  in  flight.  To  do  it 
well  requires  careful  planning,  preparation,  skill,  and  coordination.  Without  these 
things,  you  are  likely  to  painfully  rediscover  Rummler  and  Brache’s  lessons. 
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Carneg-*  tMHpn  Unr/?'af, 

Software  Engineefing  Institute 


IDEAL:  Acting 


Adoption  implementation 

•  Transform  resistance  into  support 
Skill  development 

•  Assist  people  acquire  their  new  skills 
Pilot  testing 

•  Wrong  way  to  determine  feasibility 

•  Where  do  we  need  to  improve  our  adoption  plan? 

•  What  else  can  we  produce  to  speed  adoption? 
Organizational  rollout 

•  Isolated,  phased,  all  at  once? 

•  How  long  will  the  change  last? 

•  Which  groups  are  in  the  right  place  in  their  project's 
life  cycles? 


e  mi  by  Canwsla  Malton  UnhraraSy  Vwiton  1  0  Tachnotogy  Changa  UaiMftnMnt  •  |»*g«  1} 


A  real  improvement  is  a  real  project  that  needs  real  resources,  a  budget,  project  man¬ 
agement,  executive  oversight,  and  all  of  the  other  things  that  true  mission  critical  pro¬ 
jects  need. 

If  this  improvement  isn’t  important  enough  to  have  these  critical  resources,  why  even 
start? 

An  improvement  implemented  by  junior  people  or  overloaded  people  is  likely  to  have 
problems. 

Working  on  your  organization’s  processes  is  like  working  on  your  brain.  You  really 
do  want  the  very  best  that  money  can  buy.  If  other  things  are  more  important,  then 
you  must  not  be  that  sick!  Wait  until  your  priorities  are  better,  because  a  poorly  im¬ 
plemented  improvement  is  likely  to  make  things  worse  than  they  are  and  will  cost 
more  to  repair  than  to  do  right  the  first  time. 
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Csmegie  Mellon  Untversiiy 

Software  Engineering  Institute 


IDEiAL:  Leamirg 


Reflection  on  previous  efforts 

•  What  worked  and  how  do  we  repeat  it? 

•  What  didn't  work  and  can  we  avoid  a  repetition? 


Reflection  in  action 

•  What's  working  and  how  do  we  enhance  It? 

•  What's  not  working  and  what  can  we  change? 

Reflection  on  this  effort  with  an  eye  on  the  future 

•  What  worked  and  why? 

•  What  didn't  work  and  why? 

•  What  is  likely  to  occur  again? 

•  What  changes  should  we  make  now? 
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Too  often,  people  appear  to  have  the  attitude  that  once  we  take  care  of  a  few  major 
problems,  things  will  be  okay  and  we  all  can  go  back  to  a  more  relaxed  pace. 

We  don’t  see  this.  The  speed  of  new  technology,  size  of  new  applications,  and  the 
pressure  for  shorter  cycle  times  are  growing. 


The  winners  are  those  who  recognize  that  success  will  go  to  those  who  master  obtain¬ 
ing  predictable  benefit  from  constant  change,  punctuated  by  periods  of  radical 
change.  Anything  less  positions  you  for  some  startup  to  pass  you  by  in  ways  that 
leave  you  unable  to  respond. 


IBM  was  so  focused  on  its  mainframe  business  that  it  was  not  careful  about  how  it 
negotiated  deals  with  its  software  and  chip  vendors.  If  it  had  appreciated  what  the  PC 
would  become,  Microsoft  and  Intel  would  be  divisions  of  IBM  today. 

How  many  times  have  large  firms  been  overtaken  by  small  startups?  This  is  not  a 
new  phenomenon. 
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r  Sottware  Engineering  Institute _ 

Nolhirg  h^pens  without  people 

Once  you  know  what  needs  to  be  done  and  how  you 
are  going  to  do  it,  the  challenge  changes  to  the  people 
issues. 

How  are  they  likely  to  respond  to  the  change  and  what 
will  it  take  to  convince  them  that  the  change  is  worth 
the  effort? 

How  do  you  earn  their  commitment  to  the  change  so 
they  will  use  their  creativity  to  assist  you  make  it  work 
as  opposed  to  using  their  creativity  to  make  it  go 
away? 
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It  is  critical  to  realize  that  most  workers  in  creative  jobs  have  very  little  insight  into 
how  work  flows  through  the  organization  and  how  interdependent  we  all  are. 

While  people  often  have  job  titles,  their  job  descriptions  seldom  describe  what  they 
really  do,  with  whom  they  interact,  what  things  are  received,  and  how  those  things 
are  processed  to  produce  those  things  that  are  delivered  to  others. 

Most  workers  have  defined  their  own  jobs  and  have  entered  into  a  host  of  relation¬ 
ships  and  commitments  that  are  not  clear  to  anyone  else  and  maybe  not  even  to  them¬ 
selves.  (This  is  the  “White  Space  on  the  Organization  Chart”  about  which  Rummler 
and  Brache  write.)  A  most  awkward  challenge  is  that  most  workers  would  be  hard 
pressed  to  list  all  of  the  things  they  do,  all  of  the  people  they  support,  and  all  of  the 
commitments  they  have  made  and  must  honor.  They  are  successful  because  they  take 
cues  from  everywhere  to  remind  them  of  things  to  do  and  commitments  to  honor. 

Most  people  are  event  driven  and  the  events  that  trigger  their  actions  are  subtle  and 
difficult  to  quantify. 

Introducing  a  major  change  in  an  organization  has  the  potential  to  disrupt  all  of  this. 
Therefore,  care  must  be  taken  to  address  all  of  the  obvious  things  and  be  open  to  sur¬ 
facing  and  resolving  those  issues  that  are  more  difficult  for  both  you  and  others  to 
see. 

Often,  the  only  sign  of  these  commitments  is  a  sense  of  nervousness  or  anger  for  no 
apparent  reason.  In  fact,  the  workers  may  not  be  aware  what  is  causing  their  reac¬ 
tions  and  feelings.  The  change  team  must  be  prepared  to  detect  these  signs  and  work 
the  situation  to  a  mutually  acceptable  solution. 
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Different  people  react  to  change  differently  at  different  times.  Work  by  Rogers  has 
pointed  out  that  there  are  five  distinct  categories  of  reactions  to  a  new  technology 
adoption. 

Which  group  of  people  do  you  need  to  target  at  what  step  in  your  product  develop¬ 
ment  and  deployment  to  be  successful?  Each  groups  tends  to  be  different  and  each 
group  tends  to  look  in  different  places  for  new  ideas,  inspiration,  and  information. 

During  very  early  research  phases  of  a  project.  Innovators  can  be  very  helpful  col¬ 
laborators.  During  the  big  push  for  market  share,  you  need  to  target  the  Early  Major¬ 
ity. 

Early  Adopters  are  a  most  desirable  group,  but  they  are  not  a  large  part  of  the  market 
and  their  interest  and  success  with  the  product  may  not  be  helpful  in  convincing  the 
Early  Majority  to  adopt. 

The  problem  of  moving  from  Early  Adopters  to  the  Early  Majority  was  recognized  by 
Geoffrey  Moore  in  his  book.  Crossing  the  Chasm.  This  book  has  focused  a  great  deal 
of  community  attention  on  Rogers’  earlier  work. 
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r  Software  Engineering  Institute 

Overview  of  Rogei^  s  GiorqDS 

Innovators  •  Love  to  play  with  new  things  and  seldom 
accomplish  anything 

Early  adopters  -  Able  to  see  new  potentials  and  willing 
to  act  without  much  proof  or  support 

Early  majority  -  Able  to  see  new  potential,  but  need  a 
little  proof  and  reasonable  support 

Late  majority  -  Have  to  be  shown  the  potential  benefits 
and  the  real  risks  of  not  adopting,  needs  lots  of  proof 
and  support 

Laggards  -  Have  to  be  dragged  kicking  and  screaming 
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Innovators  love  to  play  with  new  technology.  They  like  being  the  first  to  have  made 
something  new  work.  If  there’s  too  much  known  about  the  technology  or  too  many 
people  playing  with  it,  their  interests  are  drawn  to  other  things.  Another  challenge 
with  Innovators  is  that  their  work  seldom  results  in  something  that  can  be  used  to 
convince  others  to  try  the  technology. 

Similar  to  Innovators,  Early  Adopters  like  to  employ  new  technology,  but  they  tend 
to  be  more  goal  oriented.  These  people  are  talented  at  seeing  new  connections  and 
don’t  require  a  lot  of  support  to  make  it  work.  Some  of  these  adoptions  can  be  used 
to  help  convince  others  of  the  value  of  a  technology,  but  there  may  not  be  enough  of 
the  right  kind  of  data  that  the  majority  requires  and  Early  Adopters  often  work  for 
organizations  that  are  not  considered  to  be  “mainline”  by  the  majority. 

Early  Majority  Adopters  are  like  Early  Adopters  in  their  interest  in  employing  new 
technology,  but  they  often  require  more  proof.  They  are  less  able  to  be  successful 
without  support  and  training  and  may  go  elsewhere  if  these  items  are  missing.  When 
they  are  successful,  they  are  often  excellent  sources  of  data  to  convince  others. 

Late  Majority  and  Laggards  are  usually  more  work  than  they  are  worth. 
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Carnegie  Merton  Univefsity 

Software  Er>gineerlng  Institute 


Why  we  focus  on  two  groi^js 


Early  adopters 

•  Wilting  to  adopt  incomplete/immature  technologies 

•  Usually  able  to  fill  in  missing  parts 

•  Many  are  willing  to  share  their  enhancements 

•  A  source  of  data  -  but  not  credible  to  many 


Early  Majority 

•  Needs  more  of  the  whole  product,  but  not  all  of  It 

•  Less  willing  to  fill  in  the  holes 

•  A  source  of  credible  data 

•  References  from  these  early  majority: 

-  can  sway  other  early  majority 

-  may  sway  some  late  majority 
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Bringing  new  technologies  and  products  to  a  crowded  and  overloaded  marketplace  is 
not  trivial.  In  fact,  one  of  your  biggest  competitors  is  apathy.  The  immediate  cost 
savings  of  not  changing  anything  is  hard  to  beat  when  everyone  is  so  overloaded. 
Finding  Early  Adopters  means  you  don’t  have  to  invest  a  great  deal  of  time  on  build¬ 
ing  up  volumes  of  success  stories,  each  with  a  careful  ROI  analysis.  Early  Adopters 
are  desirable  because  they  usually  require  very  little  training  and  customer  support. 

Moreover,  they  may  actually  help  you  fix  some  of  the  problems  with  your  initial  re¬ 
lease. 

There  are  only  two  major  drawbacks  to  a  business  strategy  that  focuses  on  Early 
Adopters;  there  aren’t  enough  of  them  for  the  kind  of  growth  that  many  firms  desire 
and  they  seldom  stay  with  any  one  technology  long  enough  for  a  firm  to  earn  back 
their  development  investments. 

Real  market  success  depends  on  crossing  the  chasm  over  to  the  Early  Majority. 
Moore’s  book  describes  how  to  do  this  and  motivates  this  well. 
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Growing  amount  of  commitment 


In  their  paper  “Building  Commitment  to  Organizational  Change”  Conner  and  Patter¬ 
son  point  out  a  sequence  of  steps  that  must  be  taken  in  order  to  bring  about  a  change 
and  the  growing  commitment  needed  to  take  each  next  step. 

It  takes  almost  no  commitment  to  something  to  be  contacted  by  someone  or  to  be¬ 
come  aware  of  something  new. 

It  takes  some  degree  of  effort  on  a  person’s  part  to  move  from  the  state  of  awareness 
to  the  point  of  understanding.  To  expend  this  effort  requires  some  degree  of  com¬ 
mitment. 

It  takes  even  more  commitment  to  move  to  a  point  where  a  person  is  willing  to  try 
something  new  at  work  in  the  context  of  all  of  the  other  things  going  on. 

The  decision  to  move  past  a  trial  usage  to  true  adoption  requires  even  more  commit¬ 
ment. 

The  changes  needed  to  move  from  adoption  to  institutionalization,  where  people  do  it 
that  way  because  that’s  just  the  way  it  is  done  and  they  have  a  difficult  time  thinking 
of  any  other  way  of  doing  it,  requires  even  more  commitment.  Too  often  people  be¬ 
lieve  they  have  “institutionalized”  a  change,  only  to  have  it  evaporate  with  the  depar¬ 
ture  of  one  or  two  people.  A  truly  institutionalized  change  cannot  be  so  easily  lost. 
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To  maximize  the  probability  of  success  for  a  new  product,  a  number  of  supporting 
mechanisms  need  to  be  considered.  Each  mechanism  has  a  range  of  usefulness  across 
the  Conner  commitment  curve  and  the  kind  of  people  who  can  benefit  from  it.  Dif¬ 
ferent  people  play  different  roles  in  the  organization  being  changed.  These  different 
roles  will  require  a  potentially  different  collection  of  mechanisms  or  versions  of  the 
same  mechanism.  (Experienced  professional  pilots  require  different  documentation 
from  student  pilots,  but  they  must  be  based  on  the  same  underlying  procedures  and 
methods.) 

No  product  developer  can  produce  customized  mechanisms  to  address  all  of  Conner’s 
phases,  all  of  Roger’s  groups,  and  all  of  the  different  roles  played  by  all  of  the  differ¬ 
ent  people  with  their  unique  backgrounds  and  experiences. 

The  challenge,  therefore,  is  to  figure  out  which  subset  of  these  mechanisms  is  crucial, 
what  subset  of  all  possible  mechanisms  provides  the  best  ROI,  and  how  to  partner 
with  others  to  spread  the  burden  of  the  development  around. 

Distribution  partners,  value-added  resellers,  and  others  will  often  invest  their  own 
money  to  enhance  or  create  new  mechanisms  if  they  see  how  such  an  investment  will 
increase  their  own  chances  of  success.  If  the  opportunities  are  presented  properly,  the 
original  product  developer  only  has  to  produce  just  enough  of  the  right  mechanisms 
and  others  will  produce  the  rest. 
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Software  Engineering  Institute 


A  worliiy  goal 


Indeed,  one  of  my  major  complaints  about  the  computer  field  is  that 
whereas  Newton  could  say, 

"If  I  have  seen  a  little  farther  than  others  it  is 
because  I  have  stood  on  the  shoulders  of  giants,” 

I  am  forced  to  say. 

Today  we  stand  on  each  other's  feet." 

Perhaps  the  central  problem  we  face  in  all  of  computer  science  is 
how  we  are  to  get  to  the  situation  where  we  build  on  top  of  the  work 
of  others  rather  than  redoing  so  much  of  it  in  a  trivially  different 
way.  Science  is  supposed  to  be  cumulative,  not  almost  endless 
duplication  of  the  same  kind  of  things. 

R.W.Kmnlng 
On*  libn's  VWw  of  Compvtor  tcionoo 
11M  Turinf  Award  Lodurt 


O  loot  by  Camogtt  ItiliBnUntMroAr  Varahxtl.O 


Taking  new  products  to  market  is  not  new.  People  have  been  doing  this  successfully 
for  hundreds  of  years.  The  same  is  true  of  issues  in  addressing  and  managing  change. 

Unfortunately,  most  software  developers  have  no  formal  (or  even  informal)  back¬ 
ground  in  any  of  these  areas.  The  lack  of  respect  many  developers  have  for  the  soft¬ 
ware  developers  who  have  come  before  them  is  only  surpassed  by  their  lack  of  re¬ 
spect  of  people  in  marketing,  sales,  and  management  and  people  trying  to  leverage 
the  lessons  about  change.  (It’s  been  stated  that  most  software  developers  have  never 
seen  a  line  of  code  that  they  didn’t  think  they  could  written  better.) 

Since  many  people  in  middle  to  executive  management  positions  in  software¬ 
intensive  organizations  were  promoted  from  the  ranks,  it  is  not  wise  to  assume  the 
standard  business,  marketing,  sales  lessons,  and  change  management  are  known  or 
are  understood. 

Similarly,  little  is  to  be  gained  by  insulting  these  people.  Our  challenge  is  to  help 
them  appreciate  the  opportunities  and  risks  with  the  options  they  face  and  support 
them  to  the  point  where  they  can  make  an  informed  choice  and  successfully  lead  their 
organizations. 
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Appendix  C  Value  Networks 


The  value  network  is  a  graphic  representation  of  all  of  the  organizations,  groups,  and  indi¬ 
viduals  that  are  or  could  be  involved  in  the  development,  marketing,  and  use  of  a  technology. 
The  value  network  is  derived  from  the  value  chain  concept'*. 

Traditionally,  the  value  chain  [Botkin  &  Matthews  1992]  is  used  to  describe  the  process  by 
which  a  new  idea  gets  to  market.  ‘The  value  chain  is  a  sequence  of  activities  during  which 
value  is  added  to  a  new  product  or  service  as  it  makes  its  way  from  invention  to  final  distri¬ 
bution.  When  a  commercially  valuable  idea  takes  forever  to  get  from  concept  to  market¬ 
place— or  never  arrives — the  problem  is  often  a  weak  or  missing  link  (p.  26).”  The  value 
chain  is  composed  of  several  linked  stages,  which  can  then  be  grouped  into  three  phases: 

•  Phase  1:  research,  development,  design 

•  Phase  2:  production  (manufacturing,  fabrication) 

•  Phase  3:  marketing,  sales,  distribution 

One  key  way  to  navigate  the  value  chain  is  through  partnerships.  Ideally,  companies  special¬ 
izing  in  one  phase  of  the  value  chain  would  partner  with  other  companies  able  to  complete 
another  phase  of  the  process.  For  example,  large  businesses  may  be  weak  innovators  and/or 
slow  in  getting  products  to  market;  nonetheless,  these  bigger  corporations  can  offer  smaller 
partners  “stability  and  credibility,  established  marketing  and  distribution  channels,  and  finan¬ 
cial  resources  that  are  almost  unimaginable  to  strapped  young  companies  (p.  32).” 

Determining  the  value  chain  for  SEl  technologies  is  slightly  different,  since  most  SEI  tech¬ 
nologies  are  not  a  commercial-grade  product  entering  the  marketplace.  However,  activities 
with  partners  can  help  to  transition  a  technology  into  widespread  use.  In  considering  the 
value  network  for  a  technology,  partnerships  may  be  sought  for  the  following  reasons: 

•  revenue  generation,  funding  for  building  additional  elements  of  the  whole  product  for 
majority  users 

•  in-kind  resources,  a  variant  to  revenue  generation,  as  above 

•  speed  and  efficiency,  partnerships  that  decrease  the  time  to  widespread  use 

•  influence,  partnerships  with  key  players  and  opinion  leaders  (whom  others  reference  and 
follow) 


'*  Botkin,  J.  &  Matthews,  J.  Winning  Combinations:  The  Coming  Wave  of  Entrepreneurial  Partnerships 
Between  Large  and  Small  Companies.  New  York;  John  Wiley  &  Sons,  Inc.,  1992. 
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Four  major  players  are  critical  in  the  development  and  transition  of  most  SEI  technologies. 
These  organizations  are  expected  to  have  early  involvement  with  the  SEI  technology: 

•  The  SEI  technology  team  (listed  as  <technology>  in  the  figure) 

•  Collaborators 

•  Value-added  distribution  partners 

•  Other  (non-<technology>)  technology  developers 


This  table  summarizes  the  characteristics  of  these  network  entities. 


Value  Network 

Category 

Brief  Description 

Examples 

Involvement 

Technology  team 

Responsible  for  devel¬ 
opment,  maturation,  & 
transition  of  the  technol¬ 
ogy  into  the  community 

<insert  examples  for 
your  technology> 

Early 

Collaborators 

External  parties  who 

Early 

have  invested  resources 

<insert  examples  for 

;  ■  V:  ’  '  '  'V 

f  (skills  or  ftindihg)  for 

your  technology> 

••developipent  &  matura- 

Hnn  of  fhft  tftr.hnnlnav 

Value-added  Distribution 
Partners 

External  parties  who  see 
the  technology  as  valu¬ 
able  &  are  willing  to  pay 
(in  kind  or  funding)  for 
use  of  the  technology 
Primarily  involved  with 
applying  the  technology 

SEI  Transition  Partners 
who  include  the  tech¬ 
nology  in  their  offer¬ 
ings 

Early 

Other  (non- 

Orgs  developing  new 

rl’''  •; ^ ■■  ■C-  ■' ' V; f; 

<technology>)  Technol¬ 

i  product  who  may  bene¬ 

ogy  Developers 

fit  from  use  of  the  tech¬ 

nology  to  deploy  support 

I-- 

y ■  ‘''y ' 

technologies 

SEI  Business  Units 

Enabling  portions  of  SEI 
that  support  matura¬ 
tion/transition  of  tech¬ 
nologies  without  actually 
being  on  the  team 

-Licensing 

-Events  Management 

Mid-term 

Authorizing  Sponsors 

Intertial  &  external  roles 
^who  determine  how  re¬ 

S^SEI  Director’s  Office 
rSEI  Joint  Program  - 

Mid-term 

End  Users 

sources  will  be  allocated, 

(  including  how  much  will 
be  allocated  to  the  tech¬ 
nology 

Parts  of  an  organization 
that  adopt  technologies 
on  a  regular  basis 

Office 

Early  (via  Value- 
Added  Partners)  or 
Mid-term 
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